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Introduction 



As part of the test development process, this technical report is intended to present technical 
information from the tryout and pilot stages of the Michigan High School Proficiency Test (HSPTy 
in Communication Arts: Writing. There are four major parts to this report. Part 1, Evolution of 
the HSPT in Communication Arts: Writing, introduces the purpose, the legislation, and the 
committees involved in the test development. Development of the writing assessment framework 
and the framework structures is briefly described in this part. Part 2 provides an overview of the • 
exercise development of the test. Part 3 summarizes the process used in sampling, the ttyout 
design, the rating process for constmcted-response questions, reader reliability, test statistics and 
analyses, and other technical issues for the HSPT in Communication Arts: Writing tryout and pilot 
administrations. Summary results from student and teacher surveys conducted during the tryout 
stage are included in Part 4. The relevant data tables are furnished in the appendices. Operational 
technical reports will follow a similar format. 



1 . Evolution of the HSPT in Communication Arts: Writing 

The Purpose of the Michigan High School Proficiency Test 

As required by law. The Michigan High School Proficiency Test (HSPT) was developed to 
provide students with an opportunity to cam state endorsement of the local diploma. Public Art 
1 18 (P.A. 118) of 1991, Section 104(a)(subsection 7) of the School Aid Act states: 

Not later than July 31, 1993, the department shall develop and the state shall 
approve assessment instmments to determine pupil proficiency in communication 
arts, mathematics, science and other subject areas specified by the state board. 

The assessment instruments shall be based on the state board model core 
curriculum outcomes. Beginning with graduating class of 1997, a pupil shall 
not receive a high school diploma unless the pupil achieves passing scores on the 
assessment instmments developed under this section. 



The legislation initiating the development of the HSPT was introduced to respond to educators’ and 
employers’ concern that Michigan students were leaving high school without the knowledge and 
skills necessary to lead productive lives. Additionally, the high school diploma was awarded on 
the basis of local requirements. There was no consistency from school to school, nor were there, 
with the exception of one semester’s instmction in civics, state requirements for receiving a high 
school diploma. The HSPT provides a consistent measure of what students should know and be 
able to do at the end of the tenth grade in Michigan schools. 



The Expert Panel 

The Expert Panel on the Michigan High School Graduation Test was convened to advise the 
Michigan State Board of Education on important issues surrounding the high school proficiency 
examination enacted by P.A. 1 18 of 1991. The panel consisted of national experts with first-hand 
knowledge and experience in large scale testing programs (see Appendix A for list of Expert Panel 
members). 

The Expert Panel met over three days in Febmary and March of 1992 to examine the educational, 
technical, legal, fiscal and logistical issues relating to competency testing and the steps to be taken 
in the implementation of P.A. 118. Its report “Issues and Recommendations Regarding 
Implementation of the Michigan High School Graduation Tests’’ was issued in April of 1992. The 



report included 5 1 recommendations and rationale for each of the recommendations (see Appendix 
A). 



Legislation Change 

Between the issuance of the Expert Panel Report and the development of the Assessment 
Frameworks for each of the content areas tested by the HSPT, new legislation was passed which ' 
dramatically changed the intent of the test. Whereas P.A. 1 18 had stated that the awarding and 
denying of high school diplomas would be determined by HSPT scores. Public Act 335 of 1993 
softened the intent of the test. P.A. 335, Section 1279 states that the HSPT would be used to 
award state endorsements of the local high school diploma: 

Beginning with pupils scheduled to graduate in 1997, if a pupil achieves the 
academic outcomes required by the state board, as measured by an assessment 
instrument developed under subsection (8), for a state-endorsed high school 
diploma in 1 or more of the subject areas of communications skills, mathematics, 
science, and, beginning with pupils scheduled to graduate in 1999, social 
studies, the pupil’s school district shall award a state endorsement on the pupil’s 
diploma in each of the subject areas in which the pupil demonstrated the required 
proficiency. A school district shall not award a state endorsement to a pupil 
unless the pupil meets the applicable requirements for the endorsement, as 
described in this subsection. A school district may award a high school diploma 
to a pupil who successfully completes local district requirements established in 
accordance with state law for high school graduation, regardless of whether the 
pupil is eligible for any state endorsement... The assessment instruments shall 
be based on the state board model core academic curriculum outcomes... 

The change in the law also changed the context in which the Expert Panel Recommendations were 
considered in the development of the HSPT. In addition to the Expert Panel Report, several policy 
decisions and subsequent policy actions shaped the development of the HSPT from the onset. 

• The HSPT would align with the Michigan Model Core Curriculum Outcomes (State Board of 
Education, 1991), broad outcomes to be achieved by all students as a result of their school 
experiences. Fundamental to the Model Core Curriculum is the belief that the ultimate purpose 
of education is to permit each individual student to reach his or her optimum potential, to lead a 
productive and satisfying life (The Common Goals of Michigan Education, 1980). 

• The HSPT would establish high expectations for all students. 

• The HSPT would focus on the application of knowledge, problem solving and critical 
thinking. 

• The HSPT would assess what students should know and be able to do by the end of tenth 
grade. 

• Recognizing that what gets tested, gets taught, the HSPT would, to the extent possible in large 
scale assessment, model good instructional practice. 

Students earning proficient scores on the Michigan High School Proficiency Test in mathematics, 
science, writing and reading earn the state endorsement of the local diploma in mathematics, 
science and communication arts. 

Table 1 and Figure 1 show the timeline and the process used by the Michigan Department of 
Education, Michigan Educational Assessment Program (MEAP) for the development of the HSPT. 
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Table 1. HSPT Development Timeline 



Nigh School Proficiency Test 

Timeline 1992-1997 

Mathematics, Science, Reading, Writing 


1992-1993 


DeHne Test Frameworks 


November 2, 1992 


Met with MRA, MSTA, MCTM and MCTE to discuss 
Frameworks development 


January 8, 1993 


Proposals to Michigan Department of Education 


February, 1993 


Input: Preliminary Field Review by Professional 
Organizations 


March 31, 1993 


Framewords due to Michigan Department of Education 


April 21, 1993 


Michigan State Board of Education receives Frameworks 


April 221 - May 31, 1993 


Field Review and Comments 


Summer, 1993 


State Board of Education Approves Frameworks 


1993, 1994, 1995 


Test Development 


Summer 1993 
November 1993 
January 1994 


Issued RFPs 

Item/Exercise Development-Writing Test 
Item/Exercise Development-Mathematics, Science, 
Reading 


April 1994 


Tryouts-Writing 
Scoring, Analysis and Revision 


November 1994 
November 1994 
AprU 1995 


Pilots-Wiiting 
Scoring and Analysis 
Tryouts-Mathematics, Science, Reading 
Scoring, Analysis and Revision 
Scoring, Analysis 


1996-97 


Test Administration Timeline 


Spring 1996 


Test Administration 


Fall 1996 


Retest 


Spring 1997 


Test/Retest 

Award Endorsements Based Upon Results 
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Developing the Assessment Framework to Guide 
the Development of the HSPT in Communication Arts: Writing 

In 1985, the Michigan State Board of Education adopted the Michigan Essential Goals and 
Objectives for Writing. Michigan law. Public Act 25, requires that the Model Core Curriculum 
Outcomes and the Michigan Essential Goals and Objectives serve as the curriculum foundation for 
the HSPT. Both of these documents provide the basis for the framework. The Assessment 
Framework for the Michigan High School Proficiency Test in Communication Arts; Writing was ' 
developed by the Michigan Council of Teachers of English (MCTE) under contract with the 
Michigan Department of Education. The Framework Committee consisted of Michigan classroom 
teachers, curriculum coordinators, composition specialists, school administrators, testing 
specialists, and teacher educators. Michigan Department of Education staff assisted in the 
framework development. A broad representation of Michigan's diverse population was involved 
with the project. 

On April 21, 1993 the Michigan State Board of Education received the Framework developed by 
MCTE and authorized it to be disseminated by the Superintendent of Public Instruction for field 
review and comment. The Framework represents the initial work done by MCTE with revision by 
the Michigan Department of Education based upon review and comments. 

The Framework describes the writing students can be expected to do, the assessment plan for the 
exam, and task specifications for testing writing. It was prepared with a wider audience in mind, 
specifically Michigan's classroom teachers and administrators whose students are tested with the 
HSPT. 

Based on the descriptions of the Core Curriculum and Essential Goals and Objectives for Writing, 
the working definition of writing for this document is as follows: 



WRITING is 

reflecting and exploring ideas and feelings; 
creating knowledge and meaning; 
communicating ideas; and 
validating learning. 



The Assessment Framework for the HSPT in Communication Arts: Writing was based upon the 
assumptions that: 

• writing assessment should reflect the recursive nature of writing as process and the parts of 
the process that lead to finished written products: prewriting, drafting, revising; 
proofreading and publishing (see figure 2); 

• writing can only be assessed by asking writers to compose actual text; and 

• writing is important in all subject areas, not just in English language arts classes. 



Figure 2. Writing As Process 
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Prepared for the Michigan High School Proficiency Test Framework by the 
Michigan Council of Teachers of English (1993) 



Committees Involved in the Development of the 
Michigan High School Proficiency Test (HSPT) 

The Technical Advisory Committee (TAC) 

After the Expert Panel submitted its recommendations for implementing the HSPT, a subset of six 
core panel members was selected to form the Technical Advisory Committee (TAC) to serve in an 
advisory capacity during test development and implementation. Additional membership has been 
determined on an ad hoc basis as needed for particular expertise. The TAC has met with Michigan •' 
Educational Assessment Program (MEAP) staff four times or more a year to provide continuous 
advice on technical, policy and legal issues related to the MEAP tests. 

Prior to the first meeting, each TAC member received executive summaries of the assessment 
frameworks in mathematics, science, reading and writing; and portions of the proposal submitted 
by ACT, the vendor chosen to coordinate item development for writing. The TAC played an active 
role throughout test development and standard setting: shaping and reviewing plans, advising staff 
on the appropriate analyses to require of the contractor and reviewing analyses provided. The TAC 
has been intimately involved in the program at every step and continues to be involved. 

The Exercise Development Team (EDT) 

The Exercise Development Team for Writing was made up of seven Michigan teachers who were 
nominated by MDE Curriculum and MEAP staff. Members of the EDT signed a contract before 
item writing began. All members received item writing training from ACT. The primary 
responsibilities of the EDT were to develop prompts, identify stimulus materials and develop 
scoring rubrics for all tryout and pilot forms. More information about exercise development for the 
HSPT is contained in a later section of this manual. 

The Content Advisory Committee (CAC) 

The Content Advisory Committee for Writing was responsible for the integrity of the HSPT in 
Communication Arts: Writing. The CAC reviewed each test item to ensure that it was 
appropriately related to the Model Core Curriculum Outcomes (1991) and the Michigan Essential 
Goals and Objectives for Writing (1985), as set out in the legislation. Both of these documents 
were approved by the State Board of Education and disseminated to school districts well in 
advance of the first administration of the HSPT in the spring of 1996. Items were evaluated for 
consistency with the criteria set out in the Assessment Framework and appropriateness for 
measuring proficiency in writing for all students by the end of tenth grade. The CAC reviewed 
every test form to check for a reasonable distribution of item difficulty and for an adequate sample 
of the content area. Items were rejected or revised based upon decisions made by the Content 
Advisory Committee. 

The CAC for Writing was originally made up of thirteen members including high school and 
middle school classroom teachers, ^strict and school writing department chairpersons, and college 
writing instructors. 

The Bias Review Committee (BRC) 

The Bias Review Committee for Writing was comprised of six members from Michigan 
universities and local school districts. School district personnel ranged from administrators to 
content area consultants and to classroom teachers. BRC members reviewed every HSPT test item 
for possible bias to gender, racial or ethnic groups; religious groups; socioeconomic groups; 
persons with disabilities; older persons; and for regiond concerns. In instances where the BRC 
observed bias, the BRC was responsible for providing suggestions that made the test material as 
bias-free as possible, but did not distort or interfere with test content. 

Lists of members of the above committees are in Appendix A. 
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2. Exercise Development for the HSPT in Communication Arts: Writing 

A major portion of the work in the Michigan Educational Assessment Program has been done 
contractually. Through the Office of Purchasing, Department of Budget and Management, the 
Department of Education issues a Request for Proposals (RFP) describing the Department’s testing 
requirements. The successful bidder must meet both quality and cost criteria as part of the 
evaluation process. 

In order to meet the tight timeline required by legislation for development of the HSPT, ACT Inc. •' 
(hereafter ACT) was hired to coordinate the exercise development process for the HSPT in 
Communication Arts: Writing. ACT has years of experience in test development for national 
achievement tests, including writing tests. For the HSPT, with direction from MDE Curriculum 
and MEAP staff, ACT provided training for the Exercise Development Team (EDT) and facilitated 
the EDT meetings. In addition, ACT developed the initial writing item bank and test forms and ran 
item analyses on the tryout and pilot tests, including scoring. The ACT contract ran through the 
initial pilot process. 

ACT staff worked with MDE staff and the EDT to make final determinations of the content to be 
assessed, to finalize the process to be used to develop and review niaterials, and to draft, review, 
refine and finalize all prompts, administration manuds and scoring rubrics. 

During the first session of the EDT on November 15, 1993, members were trained in prompt 
writing. At this time it was decided that the assessment would be comprised of three parts. 
Members brainstormed possible themes and generated a number of prompts for Part 1. Prior to the 
second round of meetings, members were asked to write prompts for Parts 2 and 3. 

At the second round of meetings, prompts and stimulus materials for all parts of the writing 
assessment were reviewed and decisions were made about which prompts would yield the best 
writing from students. Committee members discussed the scoring rubrics and administrative 
directions for the assessment. Participants were asked to submit prompts for review and revision 
by ACT writing test specialists in preparation for the Bias Review Committee. 

The CAC and the BRC met in January, 1994 to review the work of the EDT. ACT supplied 
reviewers with all necessary materials for review, including the prompts and score point 
descriptions. The two groups made suggestions for improvements and achieved consensus on 
which prompts should be used for the tryout administration. 



General SpeciHcations for the HSPT in Communication Arts: Writing 

Following are the general test specifications for each of the three parts of the HSPT in 
Communication Arts: Writing. 

Part 1: Reflecting/Reporting 

Part 1 asks students to select two pieces of writing (perhaps from their portfolios) that best 
demonstrate that they are proficient writers to bring with them to the testing situation. Students are 
then asked to respond to a specified question about their own writing. 

• These pieces should have been written either during the school year in which the 
proficiency test is taken or during the previous school year. 

• Only one of the two pieces should be from an English or composition class; one or both 
may be from another curriculum area/class. 




17 



Page 8 



These pieces may be typed or handwritten and should together consist of a minimum of 
two pages and a maximum of ten total pages. 

Students can choose pieces in any appropriate text format, including poetry (at least 
twenty-five lines), satire, editorials, and parodies. 

These pieces should not include any grades, comments, or markings made by teachers 
or peers. 

For each piece selected, the teacher in whose class the writing was composed should be 
required to sign, to give the name of the class, and to certify that, to the best of her or 
his knowledge, the writing is the student’s own original composition.' For example. 

This paper was written for (name of class’) . I 

certify that, to the best of my knowledge, this writing is the 
original work of this student. 

f signature of teacher") 

Only when students present the two selected pieces of previous writing should they be 
permitted to take the remaining portions of the HSPT in Communication Arts: Writing. 



Task Specifications: 

• This task should be written without consultation with peers. 

• At least 30 minutes should be provided for students to respond to this task. 

• It should not be written on the same day as Part 2 or 3. 

• It should be scored as single-draft writing. 

• The prompt for this part could ask students to write about their own writing and thinking 

processes in general and/or specifically about how one or both of the pieces in Part 1 were 
written. 

Part 2: Communicating Meaning: Impromptu 

Students are provided a topic and some brief materials, such as questions, proverbs, cartoons, and 
photographs related to that topic. These materials are intended to engage students from a variety of 
backgrounds and enable them to spark prior knowledge and experiences. All materials must be 
actual works, not summaries or passages converted for the test. 

Task Specifications: 



This task should be completed in 20 minutes. 

Following the reading and/or viewing of the brief selections, students should be asked to 
write a response to the theme or to the material read and/or viewed. 



' This requirement was dropped before the HSPT in Communication Arts: Writing became operational. 



• The response may be. unpolished and exploratory, and will be scored as single-draft 
writing. 

• Students should discuss in small or large groups the material read and/or viewed and their ■ 
responses in preparation for Part 3. Students could be given a few questions to serve as 
discussion starters, if needed. 

Part 3. Communicating Meaning: Extended 

Part 2 is a warm-up for Part 3. Part 3 asks students to respond to a topic that is connected in some 

way to the topic in Part 2. 

Task Specifications : 

• This task should be completed in 1 10 minutes. 

• In order to standardize the testing conditions across the state, test monitors (preferably 
English classroom teachers) are provided a “script” in the Administration Manual which 
gives directions throughout the testing period. 

• Students have access to dictionaries, thesauri, i.e., normal classroom resource 
materials. 

• The writing produced in this task may be first- or second-draft writing but may not be 
as extensively revised, edited or proofread as it would if written in an untimed setting. 

Prompt Specifications for Part 3 Writing Task : 

• The prompt should be engaging to students, regardless of gender, socioeconomic, or 
cultural background. 

• It should elicit compositions in which students take a position and support it with 
convincing evidence. 

• It should suggest that students write for a general or respected adult audience. 

• It should provide students with considerable choice in how the writer approaches the topic 
and in writing format. 

• It should encourage students to write an extended response. 

• Its directions should remind students to support their positions with evidence which can be 
taken from a variety of sources, but it should not ask students to summarize, analyze, or 
otherwise reproduce ideas from the items read or viewed on Part 2. 

• Its directions should encourage students to prewrite, draft, revise, edit and proofread. 

• Its printed directions should include the scoring guide and scoring scale for students’ 
reference. 
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3. HSPT in Communication Arts: Writing Tryout and Pilot 



After the Exercise Development Teams completed items for each content area to be tested on the 
HSPT, the Content Advisory Committees and the Bias Review Committee reviewed all items. 
Tryouts were scheduled for the items that survived this initial committee review. Statistical data 
from the tryouts and pilots are part of the information used to determine which items merit further 
consideration for use on “live” or operational tests. In addition, participating teachers are asked to 
return comment sheets describing problems with the directions and/or items and noting 
administration details, such as the amount of time it took the majority of students to complete the 
test. Comments from teachers are particularly helpful in making decisions about items and test 
forms. 



Sample Design and Characteristics 

Data from the tryout and pilot administrations were obtained from a sample intended to be 
representative of high school students enrolled in tenth or eleventh grade in both private and public 
schools across the State of Michigan. Data from the samples were used for trying out the prompts 
and establishing the score scale. 



Tryout Sample Characteristics 

Data for the HSPT in Communication Arts: Writing tryout and pilot were collected using the same 
procedures. To ensure representativeness, cluster sampling combined with stratification was used 
to sample from Michigan public schools. The stratum refers to the geographic region and student 
population size (see Appendix A for stratum classifications). Schools participating in the tryouts 
were randomly sampled from each stratum roughly proportional to the population proportions. , A 
description of the gender and ethnic characteristics of the tryout sample by region are presented in 
Tables 2-5 (Appendix B). 

In anticipation that some schools would not participate in this study, many more schools were 
invited to participate than were required to achieve the targeted precision. During the recmitment, 
the number of participating schools in each stratum was carefully monitored so as to maintain the 
representativeness of the sample with respect to the stratification variables. 

Schools were asked to include all tenth and eleventh grade classes where writing was taught. A 
two-week administration window was provided to schools. Classroom teachers were asked to 
administer the test. Makeup testing for students who were absent was strongly encouraged. 

I 

Nonparticipation. One type of nonparticipation in this test development was among schools; 
not every school invited to participate did so. Attempts were made to choose the replacement 
schools from the same strata as the schools they were replacing so that the obtained sample would 
be representative with respect to the stratification variables. A second type of nonparticipation was 
among students within a participating school. Each writing assessment form was made up of three 
parts. The most common student non-response was failure to supply the two portfolio pieces 
necessary to complete the first part. Data for examinees who did not participate in all three parts of 
the writing test were not used in the analyses. Examinees with problematic records were also 
excluded from data analyses (e.g., grade level and/or test form not determinable, zero or missing 
writing assessment scores). 
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Tryout Administration 



After successful completion of the committee review process, ACT packaged the prompts into 
forms and printed an appropriate number of forms for the tryout adrninistration. Materials for the - 
tryout included examinee test booklets, administrators’ manuals and examinee answer documents. 
Participating schools tested between mid-March and mid- April. The number of students taking 
each part of each form is displayed in Table 6 below. 

Table 6: Number of Students Taking Each Tryout Form 



Form 


Part 1 


Part 2 


Part 3 


OIC 


361 


493 


450 


02C 


285 


602 


572 


03C 


200 


342 


322 


04C 


282 


456 


379 


05C 


459 


652 


582 


06C 


346 


602 


567 


07C 


438 


498 


289 


08C 


370 


715 


689 


09C 


282 


594 


534 


IOC 


449 


745 


675 


TOTAL 


3473 


5699 


5059 



Numerous calls were fielded during the tryout administration, primarily expressing concern or 
asking for clarification about the time limits; the complexity of the distribution and collection 
system; and the use of two student booklets (Early Drafts and Final Draft). 

Descriptive summaries of the results for each of the three parts of the Writing tryout administration 
are provided in Tables 7 through 9 (Appendix B). In addition to means and standard deviations for 
all ten tryout forms. Tables 7 through 9 provide estimates of interrater reliability, the percent of 
perfect agreement and the percent requiring a third reader. Each student response was scored by 
two readers. If these scores were not in perfect or adjacent agreement (within one point of each 
other), the assigned scores were resolved by a third reader. This situation rarely occurred. The 
two scores assigned by the original readers were compared in order to determine how closely 
different readers agreed on assigned scores. All data are broken down by form and by prompt. 

Table 10 provides the estimated variance components and g-coefficients fiom the tryout 
administration. The generalizability study used to collect data for writing involved five variance 
components: true differences among students in proficient writing as measured by the HSPT 
(Persons), differences in prompt difficulty (Prompts), differences due to readers on a given prompt 
(Raters: Prompts), interactions between students and prompts (Persons x Prompts) and 
interactions between students and readers (within prompts) (Persons x Raters: Prompts). 

Table 1 1 shows the correlations between the three parts of the HSPT in Communication Arts: 
Writing. The average correlations for all three parts of the tryout were in the .50s, with the highest 
average correlation occurring between Part 2 and Part 3 (.58). 

Tables 12 through 20 (Appendix B) provide score distributions for the total group, by gender and 
by ethnicity. 
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Scoring Procedures 






ACT staff pulled a random sampling of papers for each prompt from as many different schools as 
possible. These papers were distributed to range-finding participants to be read and scored before ' 
the mass scoring. The purpose of this step was to establish a standard for defining the “range” 
with sample papers for each score point. 

Range-finding was conducted separately for each of the three parts of the writing test. Range 
finding participants included ACT staff, external consultants, and CAC members. For all prompts, 
participants discussed and came to consensus on scores for as many range-finder papers as 
possible. During the process, the scoring rubrics and prompts were carefully considered and some 
suggestions for modifications were made. 

Following range-finding, ACT staff compiled training sets for holistic scoring of the three parts of 
the test. Materials included base and discussion sets for each prompt, as well as three sets for 
qualifying the readers. Base sets are sets of scored papers wiA different ranges of score points. 
They are used to familiarize scorers at the beginning of training with the scoring process as it has 
already been applied to actual student work. Discussion sets contained no scores or annotations. 
They were used for practicing and qualifying. Team leaders then lead the scorers through 
discussions to resolve any discrepancies in scoring the same set of responses. 

Eight table leaders were briefed and introduced to the prompts, rubrics and their various 
responsibilities in May of 1994. Three separate groups of ACT readers, one for each part of the 
assessment, were involved in the scoring. The groups included 62 readers and 8 table leaders. 
Training and qualifying for each part took one and one-half days. MDE staff was present to 
observe. All holistic scoring was completed by the end of May. 

Fifty readers and six table leaders received an additional day of training by ACT staff in analytic 
scoring of the Part 3 writing samples. 

Scores were sent to schools participating in the tryouts by the middle of June, 1994. 



Pilot Administration 



Based upon review of the tryout administration, ACT worked with MDE staff to refine the scoring 
guides, scoring strategies and the prompts. Upon completion of an external review process, MDE 
worked with ACT to compile the forms for the pilot administration. ACT produced and prepared 
revised prompts for administration during the pilot administration. 



The purpose of the pilot administration was to: 



produce six forms of the writing test that could be used interchangeably for the next 
three years. 

establish a score scale that would incorporate information from all three components of 
writing. Each component would have been scored by two readers on a four-point scale. If 
readers disagreed by more than one score point, the writing sample score would be 
resolved by a third reader. 



establish a score scale so that a cut point could be 
forms. 



determined that would be constant across 



er|c 
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ACT prepared the test booklets, answer documents, administrator's manuals and all supporting 
information in preparation of the pilot administration. 



Pilot Sample Characteristics 

The target population for the pilot sample consisted of students enrolled in eleventh grade in 
Michigan private and public schools. The overall sampling design followed the same design used - 
in the tryout administration. 

Again, many more schools were invited to participate than were required to achieve the targeted 
precision, and the number of participating schools in each stratum was carefully monitored so as to 
maintain the representativeness of the sample with respect to the stratification variables. 

Schools were asked to test all eleventh grade classes within a three-day administration. Content 
area classroom teaichers were asked to administer the test. For security purposes and in an attempt 
to minimize exposure of test forms, makeup testing for students who were absent was not 
encouraged. 

Based on the results of the tryout. Parts 1, 2 and 3 were assembled as eight forms and were piloted 
in December, 1994. The test format included; 

Part 1 30 minutes reflecting 

Part 2 40 minutes stimulus materials/quick write 

Part 3 110 minutes response to prompt 

Approximately 17,000 students participated in the pilot overall, with 13,000 students participating 
in ^1 three parts of the test. Table 21 below lists the number of students participating in the pilot 
by form. Schools were assigned to a particular group based on average MEAP reading scale 
scores from the previous year (see Table 22). 

Table 21 . Number of Students Taking the Writing Pilot by Form 



Form 


No. of Students 


OIC 


1699 


02C 


1716 


03C 


1639 


04C 


1923 


05C 


1645 


06C 


1554 


07C 


1656 


08C 


1471 


TOTAL 


13303* 



*Note: The total number here includes students who participated in all three parts of the pilot. 
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Table 22. Number of Students and Average MEAP Reading 
Scale Scores for Participating Schools by Form 



Form 


N 


Average MEAP Scale Score* 


OIC 


2580 


603.7 


02C 


2577 


603.6 


03C 


2579 


603.6 


04C 


2583 


603.5 


05C 


2573 


603.7 


06C 


2577 


603.4 


07C 


2580 


603.6 


08C 


2584 


603.6 


Average 


2579 


603.6 



Note: Numbers reflect the sum of the average scale scores for informational and narrative reading by 
school, averaged by group taking a form. The data are from the fall of 1993 test administration. 



Demographic characteristics of the sample for the pilot administration are presented in Tables 23-26 
(Appendix C). These include the grade 1 1 enrollment of participating schools, the number and size 
of schools by region and gender and ethnic characteristics by strata. 

Equating 

Test equating is necessary whenever one of two situations below occurs: 

1 . The tests are at comparable levels of difficulty and the ability distributions of the 
examinees taking the tests are similar. This is called “horizontal equating.” 

2. The tests are at different levels of difficulty and the ability distributions of the 
examinees are different. This is called “vertical equating.” 

For HSPT tryouts and pilots, horizontal equating was used because multiple forms were developed 
for each subject area and administered to randomly equivalent groups in the sample. The purpose 
of equating is to transform the scores of examinees taking form X to equivalent scores in form Y 
so that these scores can be compared to the scores of examinees taking form Y. 

Each student took one pilot form. Since there were 8 forms and no items overlapped between any 
two forms, randomly equivalent group equating was used. To avoid exposing dl forms in a 
participating school, forms were divided into eight groups of triplets (Table 27, next page). A 
school was randomly assigned to take only one group of forms. The forms within each triplet 
were then spiraled and administered to students within a classroom so that no students sitting next 
to each other would have the same form. This design permitted the equating of forms between 
triplets through the assumption of randomly equivalent groups of different participating schools 
taking the same form, but in different combinations. 



Table 27. Pilot Form Composition 



Group 


Forms 


1 


03C 


06C 


08C 


2 


02C 


04C 


08C 


3 


05C 


07C 


08C 


4 


02C 


05C 


06C 


5 


OIC 


02C 


03C 


6 


OIC 


04C 


05C 


7 


OIC 


06C 


07C 


8 


03C 


04C 


07C 



Nonparticipation. As with the tryout administration, not every school selected participated in 
the pilot nor did all students participate within a selected school. Again, many students failed to 
supply the two portfolio pieces necessary to complete Part 1 of the test and data for students who 
did not participate in all three parts were not used in the analyses. Examinees with problematic 
records were also excluded from the analyses (e.g., grade level and/or test form not determinable, 
zero or missing writing assessment scores). 

Analyses of Pilot Test Results 

A summary of the results for each of the three parts of the HSPT in Communication Arts: Writing 
pilot administration is provided in Tables 28-30 (Appendix C). All data are broken down by form. 
In addition to means and standard deviations, these tables provide estimates of interrater reliability, 
the percent of perfect agreement and the percent of papers requiring a third reader (% resolved) for 
each of the eight forms of the pilot. 

Tables 3 1 through 33 (Appendix C) provide the estimated variance components and g-coefficients 
from the pilot. As same as the tryout. The generalizability study used to collect data for the 
Writing test involves five variance components: tme differences among students in proficient 
writing as measured by the Proficiency Test (Persons), differences in prompt difficulty (Prompts), 
differences due to readers on a given prompt (Raters: Prompts), interactions between students and 
prompts (Persons x Prompts) and interactions between students and readers (within prompts) 
(Persons x Raters: Prompts). 

Table 34 (Appendix C) provides correlations between the three parts of the test within each form. 
The average corelations between any two of the three parts were between .55 and .57. 

Score distributions (Tables 35-43) for the total group, by gender and by ethnicity are also provided 
in Appendix C. 
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Recommendations for Scaling and Equating 

The following assumptions were made during the completion of these analyses: 



• A comparable score scale was desirable across test forms. 

• Only a composite score would be “adjusted,” where the composite was a weighted sum of 
the three parts. The composite score would be the reported (reconciled) score. 

• It made no difference which form would be used as the base form. 

• Data were edited prior to use. For example, poor data, poor test centers, and poor Judges 

would be edited out prior to analyzing the data. 

• The pass/fail decisions were based on two holistic ratings per writing sample for each of 
the three writing samples. 

• The ratings were on a four-point scale. 

• Analytic ratings were not used for pass/fail decisions. 

• Because no examinee was administered two complete forms, there was no way of 

ascertaining, even after the scores were adjusted, whether certain examinees would perform 
differently on one form than on another form. 




The comparability results based on readers from the pilot administration would be 
appropriate for use with readers for the operational administrations. (This assumption is 
viable only to the extent that the training and scoring procedures used operationally were 
similar to those used during the pilot.) 



Based on the recommendations of the Technical Advisory Committee on May 16, 1995, ACT 
completed the equating analyses for the HSPT in Communication Arts: Writing for the eight pilot 
test forms. The procedures included: 




Form 06C was used as the base form, to which all other forms were equated. One form 
needed to be chosen to serve as the anchor form; Form 06C was chosen because it was a 
"typical" form (appeared similar to other forms with which it was spiraled). 

The composite score was computed using the 1-1-2 weighting scheme, where the Part 1 
score, the Part 2 score and twice the Part 3 score were summed to obtain the composite 
score. This weighting appeared to be a good compromise between optimizing both 
statistical and measurement and content considerations. 

A score scale for the composite was set on the base form (Form 06C) by mapping the 
possible raw composite scores into consecutive integers. This results in a scale that mns 
from 1 to 25 (half points counted). 

When Forms OIC, 02C, 05C, 07C and 08C were equated to Form 06C using mean 
equating, the reported scores were on the same scale. That is, a raw score of 4 (e.g., 1 
point for Part 1, 1 point for Part 2, 1 point x 2 [weights] for Part 3 = 4 points) on each of 
these forms is reported as a scale of "1," a 4.5 is reported as a scale score of "2," etc. The 
'unrounded scale scores, those scores that would be used if a form was an anchor in a 
future equating study with new forms, do differ. 
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Because Form 04C was not spiraled in a group with Form 06C, it could not be directly 
equated. However, 04C was equated to the score scale through Form OIC, and then 
reported scores were an identity using mean equating. 

When Form 03C was equated to Form 06C, the reported scores were not an identity. This 
does not mean that there is anything wrong with Form 03C, nor that it could not be put on 
the score scale using an equating adjustment. It is, however, ACT’s recommendation that 
Forms OIC, 02C, 04C, 05C, 06C, 07C and 08C be considered interchangeable based on 
the data that were used from the mean equating results. 



Pilot Scoring Procedures 

The ACT Performance Assessment Scoring Center assumed responsibility for the activities 
associated with scoring the pilot forms. Scoring guides developed by the EDT and refined by ACT 
were used. 

ACT staff were asked by MDE to pull a random sampling of papers for each prompt for all three 
parts of the HSPT in Communication Arts: Writing from as many different schools as possible. 
These papers were distributed to participants to be read and scored in advance of the meeting for 
range-finding which took place in January of 1995. Participants included ACT staff, external 
consultants, and CAC members. For all prompts, participants discussed and came to consensus 
on scores for as many range-finder papers as possible. During the process, the scoring rubrics and 
prompts were taken into careful consideration, and some suggestions for modification were made. 

Following the range-finding meeting, ACT staff compiled training sets for scoring the three parts 
of the assessment. Materials included base and discussion sets for each prompt as well as three 
qualifying sets. 

Prior to the reading training session, a committee consisting of writing and language arts specialists 
read a substantial number of responses drawn from a field test administration. From their readings 
and discussions, papers were selected for use in the training process. Approximately 24 responses 
were selected to serve as “anchor papers” representing the range of responses demonstrated in the 
total set of papers. In addition to the anchor papers, approximately 40 responses were selected for 
discussion during the training process and 60 responses were selected for qualifying papers. 

Readers recruited were required to have completed an undergraduate degree in writing, education, 
or related area. All readers had limited or no experience in reading writing assessments and all 
readers were new to this particular scoring project. The readers were approximately half males and 
half females. 

During the training session, readers were shown the 24 anchor papers. The trainers discussed 
each response and explained the reason why each had received its assigned score. After this 
discussion, readers were given a 20-response discussion set. Readers scored the responses 
independently and recorded the scores. When finished, the trainer discussed each response and 
explained why each had received its assigned score. This process was repeated for a second 20- 
response discussion set. 

After the discussion, sets, all readers were given three sets of 20-responses for the purpose of 
qualifying for the particular scoring project. Readers were considered qualified if the score they 
assigned matched the score assigned by the writing staff on at least 60 percent of the papers and 
they were within one point on additional 30 percent of the papers. IF readers did not qualify, they 
received additional training and were then given another opportunity to qualify. Any reader who 
failed to qualify after the retraining was excused from the project. 



After the training process was complete, each response was read independently by at least two 
readers. If the scores assigned by these two readers were more than one point apart, a third, more 
experienced reader resolved the discrepancy between scores. The two scores for each paper were ' 
averaged to produce the final score for each examinee, resulting in scores between 1 and 6 in 
increments of .5. 

As the scoring project progressed, several packets of papers were circulated through the scoring 
process to monitor the ongoing performance of the readers. Scores had been previously assigned 
to these papers by the trainers. If readers drifted from the master scores, they received additional 
training. 

Eight table leaders were briefed and introduced to the prompts, rubrics and their various 
responsibilities in March of 1995. Two separate groups of ACT readers, one for each part of the 
assessment, were involved in the scoring. The groups included 32 readers and 4 table leaders for 
Part 3 and 20 readers and 2 table leaders for Parts 1 and 2. MDE staff was present to observe. 
Each student response was scored by two readers. If these scores were not in perfect or adjacent 
agreement (within one point of each other), the assigned scores were resolved by a third reader. 
The two scores assigned by the original readers were compared in order to determine how closely 
different readers agreed on assigned scores. All data are broken down by form. Training, 
qualifying and all holistic scoring was completed by April 6, 1995. 

As a result of an MDE decision, there was no analytic scoring done for the pilot administration. 

Scores were sent to schools participating in the HSPT in Cormnunication Arts: Writing pilot in 
May 1995. 
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Part 4. Student Survey and Teacher Survey 

The Technical Advisory Committee (TAG) recommended that a study be done prior to the first 
administration of the Michigan High School Proficiency Test and again just prior to the time when * 
the first graduating class would be impacted. 

In early 1994, planning for an opportunity to learn study began. It was tentatively agreed that the 
final responsibility for the design must reside at the State Department level, that members of the 
Framework Committees should be involved in the design, that teachers in every district needed to 
be surveyed, that students should be sampled, and that the TAC should review the sampling plan , 
and the draft survey instrument(s). 

In March 1994, one TAC member. Department staff, and a member of the Science Framework 
Committee reached two major decisions: 

( 1 ) Surveys would be sent to every high school to the subject matter coordinators for the 
content areas tested on the HSPT. They would be asked to form committees of teachers 
from their high schools as well as their feeder schools to fill out the survey. 

(2) A sample set of students would be part of the study. 

In subsequent meetings with the Framework Committees, discussions were held regarding the 
content and the format of the surveys. It was agreed that the general form of the surveys was to be 
the same across content areas, but that format should not take precedence over substance and if 
there were good reasons for having different formats, it would be allowed. Content area experts 
were to be responsible for the actual wording of the surveys. 

The study was originally intended to address three purposes: (1) to help make adjustments to the 
tests if necessary, (2) to aid in standard setting and (3) to provide schools with information that 
could be used for professional development. 

On September 2, 1994, an overview of the proposed design was presented to the TAC. The TAC 
members suggested that the names of the surveys be changed from "opportunity to learn" surveys 
to the "Teacher Survey" and the "Student Survey." Revisions were suggested and made for the 
Student Survey. The Teacher Survey was discussed at length, reviewed and revised. Both the 
student and teacher surveys were piloted at several sites before being sent out. 



Writing Student Survey Results 

The Writing Student Survey (see Appendix D) was given to the students who participated in the 
Writing Pilot. The students completed the survey prior to taking the item pilot "test" so that student 
perceptions pertaining to performance would not influence survey responses. 

The Writing survey contained 18 statements. The common stem was as follows: "By the end of 
tenth grade, how often did your school experience include:..." Students were to respond on a 
four-point scale from "never" to "a lot." Note that "never" was translated to a value of "zero" (0), 
"very little" to "1," "some" to "2," and "a lot" to "3." 

Table 44 below presents the summary data for the student survey results. The mean score for the 
18 writing survey questions was 1.87 (2.00 = some). The lowest mean for a survey question was 
.99, which places it about "very little." Four questions (22%) had a majority of the students 
respond less than "some." Four questions (22%) had a mean less than 1.5. By part, the mean 
survey scores ranged from a low of 1.68 for “kinds of writing” to a high of 2.26 for “writing 
process.” Student Response means for each statement are shown in Table 45 (Appendix D). 
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Because the surveys were given to the same students who participated in the pilot, it was possible 
to correlate the mean scores for the students on the survey with their scores on the pilot tests. The 
correlations are positive, but not particularly high (.2946). Thus, the students' perceptions of 
whether they were taught something did not seem very highly related to how they actually scored 
on the pilot. 

Among the content areas, the writing student survey results were between mathematics and 
science. However, it should be noted that 22% of the questions were answered less than “some” 
by a majority of students and had means less that 1.5. 

Table 44: Student Survey Results Summary 
Content: Writing 



Total 

# of questions 


18 


overall mean 


1.87 


lowest mean 


.99 


# and % of questions that 
the majority marked 
less than “some” (2.0) 


4 (22%) 


# and % of questions 
with a mean less than 1.5 


4 (22%) 


correlation statistic of 
survey mean and tryout 
score 


.29 




Conclusions From Student Survey 

In drawing conclusions from the student survey results, one must keep in mind that there was no 
good way of determining how honestly students responded to the questions or even the extent to 
which they understood the questions. Given those cautions, it was concluded that school 
experiences in general included the types of activities useful in assisting students to learn the 
content to be tested on the proficiency test. 



Communication Arts: Teacher Survey 



The Teacher Survey was sent to writing supervisors at all high schools in the state (N=758), May 
of 1995. These supervisors were each to form a team of teachers to work with them in completing 
the Teacher Survey and an Instructional/Curriculum Support Materials Form (which they did not 
need to return). The return rate was low: 245 schools in Communication Arts (32.3%). Thus, 
caution must be paid in interpreting the data below. 



The Communication Arts Teacher Survey was composed of 50 statements (24 Writing and 26 
Reading) organized by parts. The Writing parts are as follows: (a) types of writing (genre), (b) 
writing process, (c) working to improve components of writing, (d) writing in your schools, and 
(e) student papers. For Parts (a) through (d), respondents circled all grades receiving instruction. 
For Part (e), they circled the one grade at which sufficient classroom instruction had occurred to 
expect understanding/proficiency. 
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Summary Of Teacher Survey Results 



In summarizing the Communication Arts teacher survey results, it must be remembered that the 
data analyzed is based on a low return rate of 245 responses out of 758 surveys sent to schools. * 
So, the responses may not be representative. Nevertheless, some tentative findings emerge from 
the teacher survey results which are summarized in Table 46 below. Additional survey findings 
are noted in Appendix D: 

• for eight of the 24 statements, no school circled “NT’ (Not Taught); 

• no statement had more than 50% of the schools circle the “NT” response; 

• only two statements had more than 25% of the schools circle “NT”; 

• no statement had 50% or more of the schools circle “NSI” (Not Sufficient Instruction); 

• eighteen of the 24 statements had fewer than 10% of the schools circle “NT”; and 

• three out of four statements had “NSF’ circled by fewer than 10% of the schools. 



Table 46. Teacher Survey Results Summary 
Content: Communication Arts: Writing 



# and % of statements where 
NT circled by 25% or more 


2 

(10%) 


# and % of statements where 
NSI circled by 50% or more 


0 

(0%) 


# and % of statements where 
NSI circled by 25% or more 


1 

(25%) 


# and % of statements where 
NSI circled bv less than 10% 


3 

(75%) 



Overall Summary And Follow-Up^ 

Both the student and teacher survey results suggested that many of the objectives were already 
being taught in the majority of the schools and that they were sufficiently taught for students to 
have proficiency in them. 

The results of both the teacher and student surveys were presented to the standard setting 
committees at the time they made recommendations regarding scores. Prior to that time, the 
department devoted considerable time determining just how the data should be presented and what 
the committees should be told about the relevance of the data for standard setting. It must be 
stressed that these data were gathered in the 1994-95 school year, and that information about the 
content of the proficiency tests continued to be widely disseminated before the test was given in the 
spring of 1996. It is reasonable to believe that instruction in the schools has become more aligned 
to the objectives tested as time has passed. 

The results of these surveys were disseminated to curriculum coordinators in the schools who were 
encouraged to use them in planning curricular/instructional changes prior to the first administration 
of the HSPT. It should have been clearly understood by local schools that it is in the best interests 
of their students to teach them material from a content domain that is sampled on a test for which 
achieving proficiency is a requirement for a state-endorsed certificate. 



^ In July, 1996, the State Board of Education approved the standards as set by the standard setting committees, 
without changes. 
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Expert Panel* 




Mr. Thomas Fisher 

Administrator of Student Assessment Services Section 
Florida Department of Education 



Ms. Sharon Johnson-Lewis 

Director of Planning, Research and Evaluation 

Detroit Public Schools 



Ms. Maijorie Mastie 

Supervisor for Assessment Services 

Washtenaw Intermediate School District 



Dr. William Mehrens, Expert Panel Chair 
Professor of Educational Measurement 
Michigan State University 







Dr. Jason Millman 

Professor of Educational Mearsurement 
Cornell Unvei-sity 



Dr. Susan Phillips 

Associate Professor of Educational Measurement 
Michigan State University 



Dr. Edward Roeber 

Director, Student Assessment Programs 
Council of Chief State School Officers 



Dr. Roger Trent 

Director, Division of Educational Services 
Ohio Department of Education 

* Job titles at time panel convened 




34 



Page 24 



Technical Advisory Committee (TAC) 



Dr. Gail Baxter 

Assistant Professor of Education 
University of Michigan 



Dr. Roger Trent 

Director, Division of Educational Services 
Ohio Department of Education 



Ms. Sharon Johnson-Lewis 
Assistant Superintendent 
Research, Development & Coordination 
Detroit Public Schools 



Dr. William Mehrens 

Professor of Educational Measurement 

Michigan State University 



Dr. Edward Roeber 

Director, Student Assessment Programs 
Council of Chief State School Officers 



Dr. Joseph Ryan 
Research Consultant Center 
University of South Carolina 



* Job title at time of HSPT development 



Content Advisory Committee (CAC)* - Writing 



Ms. Cheryl Snell, Principal 
DeKeyser Elementary School 
Utica Community Schools 

Dr. Ron Sudol 
Professor of Rhetoric 
Oakland University 

Ms. Nancy A. Harper 
Language Arts Coordinator K-12 
Forest Hills Public Schools 

I 

Ms. Sharon Harper 

Language Arts/Special Ed Teacher 

Norup Middle School 

School District of the City of Berkley 

Dr. Ronald Kar 

Director, Office of Communication Arts 
Detroit Public Schools 

Mr. Ray Lawson 
English Teacher 
Rochester High School 

Ms. Susanna Defever 
Head of English Department 
St. Clair Community College 



Dr. Mary Ellen Tindall 
John Glenn High School 
Wayne-Westland Community Schools 

Ms. Katherine Kososki 
Westwood High School 
N.I.C.E. Community Schools 

Ms. Nancy Elliott 
Ubly High School 
Ubly Community Schools 

Ms. Barbara Rebbeck 

North Farmington High Schools 

Farmington Public Schools 

Ms. Louise Garcia Harrison 
Heritage High School 
Saginaw City School District 

Dr. Ellen Brinkley 
Associate Professor 
Western Michigan University 



* Job title at time of HSPT development 
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Exercise Development Team (EDT)* - Writing 



Ms. Lx)uise Garcia Harrison 
Heritage High School 
Saginaw School District 

Mr. Paul Wolbrink 
Spring Lake High School 
Spring Lake Public Schools 

Ms. Maryalice Stoneback 
Ogemaw Heights High School 
West Branch-Rose City Area Schools 

Ms. Mary Cox 

M. L. King High School 

Detroit Public Schools 

Mr. Ronald Iwandkovitsch 
Ubly High School 
Ubly Community Schools 

Dr. Martin White 
Department of English 
Central Michigan Lfniversity 

Ms. Dana Payne 
Renaissance High School 
Detroit Public Schools 



* Job title at time of HSPT development 



Bias Review Committee (BRC)* - Writing 



Dr. Mary Harmon 
Department of English 
Saginaw Valley State University 

Mr. Jose Flores 
Assistant Director 

Compensatory Education & Program Development 
Grand Rapids Public'Schools 

Dr. Geneva Smitherman 
University Distinguished Professor 
Department of English 
Michigan State University 

Ms. Mary L. Jackson 
Ricker Middle School 

Ms. Christina McGrinson 
Woods Lake Elementary School 
Kalamazoo Public Schools 

Dr. Shereen Arraf 

Coordinator of Testing and Evaluation 
Administration Service Center 
Dearborn Public Schools 



* Job title at time of HSPT development 



MEAP Writing Assessments in Grades 5 and 8 
High School Proficiency Test/Communication Arts: Writing 



Bias Review Committee 



Some Issues to Consider When Reviewing 
Assessment Materials for Fairness 



All assessment materials should be: 
free of stereotypes 

accessible to all students regardless of gender, 
geographic location, or cultural or ethnic background 

free of ‘weighting” toward students with certain 
experiences (e.g., pets or extensive travel) 

independent of particular viewpoints or opinions 



BIAS REVIEW COMMITTEE 
COMMENT SHEET 



MICHIGAN EDUCATIONAL ASSESSMENT PROGRAM 

MICHIGAN DEPARTMENT OF EDUCATION 



TEST ITEMS BEING REVIEWED (Content Area and Grade) 

DATE MDE Representative 

The below items were judged to be problematical by the Bias Review Committee. 



Form # 


Item # 


Bias Issue 


Comments 
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BIAS REVIEW COMMITTEE 
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Michigan Stratum Classification on Region 



Region 1 
Region 2 
Region 3 
Region 4 
Region 5 



Metropolitan Detroit (Wayne, Oakland and Macomb counties) 

Southeast Lower Peninsula 

Southwester Lower Peninsula 

Northern Lower Peninsula 

Upper Peninsula 



I 
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Expert Panel Recommendations . 

1 . The State Board should not specify subject areas other than Communications Skills, 
Mathematics, and Science for the initial assessment. 

2 . Communication skills assessed during the first assessment cycle should be limited to 
reading and writing. 

3 . The State Board and the Michigan Department of Education need to determine which 
subsets of the model core curriculum should be included in the assessments. This needs to 
be done very shortly. The decision should be based on recognition of the importance of 
students’ opportunity to leam the content and some knowledge regarding what is likely to 
be in the school curricula by the date of the first test. The decision should not be that the 
total core curriculum is the appropriate domain from which to build the tests. 

4 . Once a determination is made regarding the testable portion of the core curriculum, there 
should be an administrative rule or statute that specifies this portion of the core is exempted 
from the permissive language in P.A. 25 and must be taught by the local districts to all 
students. 

5 . Once the testable portion of the core is determined, there should be wide publicity of this to 
the local districts. Consideration should be given to how this information can be 
disseminated with enough detail to let students and educators know the knowledge and 
skills to be tested but without so much detail that the students can answer the questions 
without understanding the curricular elements from which the items are only a sample. 

6 . Gather evidence from both teachers and students regarding the opportunity to leam the 
content domain the tests sample prior to the first administration. 

7 . Provide instructional support and training to local teachers if there is a need. 

8 . The State Board should not make any changes in the core curriculum or selected testable 
core prior to 1997. 

9 . When (or if) any changes are made in the core curriculum, there must be a phase-in period, 
and the tasks described in recommendations 3 through 7 would need to be repeated. 

10. Name the assessment the “Michigan High School Graduation Tests.”’ 

1 1 . The Department of Education should caution its employees and the State Board against 
making any unsubstantiated statements about what the tests measure or what inferences can 
be made from the test scores. There should be an official statement about the tests and the 
inferences that can be drawn from the scores. 

12. Demand that the test developer design sufficient safeguards to ensure that the test 
adequately samples the defined content. 

13. Be careful not to make any official statements that would suggest the test has criterion- 
related validity if supportive data have not been gathered. 



Because there will be different tests for different content areas, we suggest the plural “tests”. However, for ease in 
subsequent writing we will, at times, refer to the total assessment as a test. When we do so, it should be understood 
that the reference includes all the tests. 



14. Contract for enough items initially so that after losses through pilot and field testing there 
will be enough to build forms through the 95-96 administration year. 

1 5 . Reissue a contract in sufficient time to have items developed and tried out (possibly 
embedded in a live form) prior to their being needed for the 96-97 year. 

16. Schedule a large scale field tryout for tenth graders by the spring of 1994. ' 

17. Appoint and train a standard-setting committee. 

1 8 . Use a technical advisory committee to help develop a specific standard-setting procedure. 

19. The State Board of Education should establish a passing score through administrative rule 
based upon a recommendation by the superintendent of public instruction with the advice of 
appropriate committees. 

20. Consider setting incremental cut scores for different graduating classes at the time the State 
Board of Education makes its initial decision. 

2 1 . The item sensitivity reviews should be completed by a committee that is selected and trained 
specifically for this task. Most members should represent Michigan’s predominant 
minority groups. However, it would be wise to have at least one member of the committee 
be a minority group member from out-of-state who is a recognized expert in the area. 

22. Statistical item bias studies should be conducted. Items which show up as statistically 
biased should be reviewed (but not necessarily discharged) by an item bias committee 
(conceivably, but not necessarily the committee used for the item sensitivity review) and a 
content review committee. 

23 . Obtain the following reliability estimates: internal consistency, inter-rater reliability, 
generalizability across writing samples, and the reliability or standard error at the cut score. 

24. Scores should be reported as “Pass” or “Fail.” Those individuals who fail should be given 
some information regarding how close they were to passing, and they should be given 
some diagnostic information that would facilitate remediation efforts. There are important 
technical details (e.g., reliability of difference scores) regarding various methods of 
reporting diagnostic information and specific plans should be formulated by a technical 
advisory committee prior to approval of the final test specifications. 

25. We would encourage use of a common scale across subject matter areas. This takes some 
advance planning to avoid adopting a scale that is appropriate for one test, but unworkable 
for another. 

26. Develop detailed rules (procedures) for designating forms for make-up examinations and 
out of school (i.e.. Adult Ed.) populations. Determine whether you should ever reuse a 
form. Determine how many times you will administer the test each year. Determine 
equating procedures (e.g., number of anchor items to be used). Based on these 
considerations, initially develop enough alternate forms to last through at least the 1995-96 
school year. Start developing more forms/items prior to that so a sufficient supply is 
continuously available. 

27. Use a technical advisory committee to help develop specific equating procedures. 
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28 . 



29. 



30. 

31. 



32. 

33. 

34. 




35. 



36. 



37. 



38. 



39. 



40. 




Consider carefully policies regarding all test administration conditions. For example, the 
decision of whether or not to use calculators in the mathematics test must be made by the 
department, not by local school personnel. Train local school personnel adequately to 
administer the tests. Consider random auditing of the administration process to ensure 
uniformity throughout the state. 

Be cautious about any “predictive” interpretation of the scores of any single individual from 
testing in earlier grades. Such tests should be thought of as providing only an early 
awareness. 

The department should prepare and have the board adopt written procedures regarding 
make-up examination provisions. 

The department should prepare and have the board adopt specific written rules regarding 
the number of retakes that should be allowed, and how many attempts a student should be 
given prior to the time he/she is scheduled to graduate. 

Develop a detailed proposal that addresses questions regarding remediation efforts and the 
respective responsibilities of the state, the district and the student for remediation efforts. 

Enact an administrative rule regarding testing issues related to special education students 
and students with limited English proficiency. 

Individuals in adult education programs who wish to receive high school diplomas after the 
end of the 1996-97 school year should be required to pass the High School Graduation 
Test. 

Obtain the services of the Attorney General’s Office early on in the process and 
continuously as new policies are developed and implemented. 

The State Superintendent of Public Instruction and the State Board of Education should 
work with the legislature to adopt statutory authority for the high school graduation testing 
program. 

Carefully investigate liability issues with assistance from the Attorney General’s Office. 
Attempt to obtain necessary statutes with respect to liability. Inform all committees and all 
staff regarding their potential liability. 

Schools should be notified immediately regarding this graduation requirement and the 
information disseminated to all teachers. Students and their parents should be notified no 
later then the spring of 1993. 

The department should prepare, and the board should adopt, detailed policies regarding 
what should be documented and how long the documentation should be kept on file. We 
generally suggest that all documentation be kept for a period of at least five years following 
the school year in which the test was administered. We suggest keeping “forever” the 
initial development documentation and records about when, why, and how procedures are 
adopted and/or changed. 

In consultation with the Attorney General’s Office, and based in part upon discussions with 
representatives of state education associations (e.g., teachers’ unions and administrators’ 
associations), the department should prepare, and the State Board of Education should 
adopt, rules regarding what constitutes inappropriate behavior on the part of educators or 
students with respect to test-taking behavior, security issues, and so forth; and what 
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penalties will be imposed for violation of these rules. These rules and the penalties should 
be disseminated to educators and students prior to the initial administration of the 
graduation test. 

4 1 . The department needs to develop a complete list of rules/regulations that need to be adopted 
and decide whether these can simply be adopted by the board or whether they need 
legislative approval. 

42. Detailed security arrangements need to be developed. 

43. Detailed policies regarding security valuations need to be established. Staff should 
investigate current laws regarding freedom of information exclusions, and if they are 
insufficient, request new legislation to exempt secure test materials from the freedom of 
information regulations. 

44. The department needs to determine what additional equipment/facilities are needed for 
storage of secure materials, shredding out-of-date secure materials, etc. 

45. An annual test administration plan should be developed and disseminated to all school 
districts. 

46. The tests should first be administered to 10th graders in the spring of 1995 and they should 
be administered at least twice each in the junior and senior years. 

47. The department should conduct a careful study to assess additional staffing needs in 
assessment and instructional programs. 

48. The position of supervisor of state assessment should be filled as quickly as possible. 

49. The following advisory committees should be appointed: 1) a Michigan Department of 
Education Steering Committee, 2) a Testing Policy Advisory Committee, 3) a Bias Review 
Panel, 4) a Technical Advisory Committee, 5) a Content Review Committee in each content 
area of the test, 6) an overall content review committee, and 7) a Standard Setting 
Committee. 

50. Use at most two contractors: one for test development and formal field tryouts; and another 
for test administration, scoring, and reporting. 

5 1 . Obtain more detailed information from other states with similar programs regarding fiscal 
needs. Make recommendations to the legislature that are sufficient to cover department 
needs, and make clear to them that the task simply cannot be done without adequate 
support. 
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Holistic Scorepoint Descriptions 
High School Proficiency Test in Writing 

These are designed to be used in conjunction with illustrative anchor/guide papers and other range-" 

finder papers and are intended to describe characteristics of most papers at a particular scorepoint. 

The aim is to determine best fit; a paper at any given scorepoint may not include all characteristics. 

Part 1 

4 The written response demonstrates the ability to reflect critically on one’s own writing; 

ideas are supported by specific examples or details from the portfolio pieces. Organization 
and form enhance the central ideas and move the reader through the text. The voice and 
tone are authentic and compelling. There may be minor surface feature errors. 

3 The written response demonstrates the ability to reflect on one’s own writing; ideas are 
somewhat supported by examples or details from the portfolio pieces. Organization and 
form are appropriate and present the ideas coherently. The voice and tone support the ideas 
conveyed. Surface feature errors may be noticeable. 

2 The written response demonstrates some ability to reflect on one’s own writing; ideas are 
presented as generalizations about writing or simple summaries of portfolio pieces. The 
voice and tone may be inappropriate or uneven. Limited control of surface features may 
make the writing awkward to read. 

1 The written response demonstrates the attempt to reflect on one’s own writing; ideas are 
supported by few, if any, details and examples. There is little discernible shape or 
direction. There is little control over voice and tone. Limited control of surface features 
may make the writing difficult to read. 

Part 2 

4 The writing is engaging, original, clear, and focused; ideas and content are richly 
developed with details and examples. Organization and form enhance the central idea(s) or 
theme; ideas are presented coherently to move the reader through the text. The voice is 
compelling and conveys the writing’s meaning through effective sentence structure and 
precise word choices. Surface feature errors don’t interfere with understanding or distract 
from meaning. 

3 The writing is reasonably clear, focused, and well-supported; ideas and content are 
adequately developed through details and examples. Organization and form are 
appropriate, and ideas are generally presented coherently. The voice contributes to the 
writing’s meaning through appropriate and varied sentence structure and word choices. 
Surface feature errors may reduce understanding. 

2 The writing has some focus and support; ideas and content may be developed with limited 
details and examples. The writing may be somewhat disorganized or too obviously 
structured. The voice of the writer is generally indistinguishable; basic sentence structure 
and limited vocabulary convey a simple message. Limited control of surface features make 
the paper awkward to read. 

1 The writing has little focus and development; ideas and content are supported by few, if 
any, details and examples. There is little discernible shape or direction. The tone is flat. 
Awkward sentence structure and inadequate vocabula^ interfere with understanding. 
Limited control of surface features make the writing difficult to read. 



Part 3 



4 The writing is engaging, original, clear, and focused; ideas and content are richly 
developed and supported by details and examples where appropriate. Control of 
organization and transitions move the reader easily through the text. The voice and tone are 
authentic and compelling. Control of language and skillfol use of writing conventions 
contribute to the effect of the presentation. 

3 The writing is generally clear, focused, and well-developed; examples and details support 
ideas and content where appropriate. The presentation is generally coherent, and its 
organizational structure is functional. The voice, tone, diction, and sentence structure 
support meaning. Use of writing conventions is not distracting. 

2 The writing has some focus and support; ideas and content may be developed with limited 
details and examples. The presentation shows some evidence of stmcture, but it may be 
artificial or only partially successful. The tone may be inappropriate or the voice uneven. 
Sentence stmcture and diction are generally correct but rudimentary. Limited control of 
writing conventions may interfere with meaning some of the time. 

1 The writing has little focus and development; ideas and content are supported by few, if 
any, details and examples. There is little discernible shape or direction. The writing 
demonstrates no control over voice and tone. Faulty sentence stmcture and limited 
vocabulary interfere with understanding. Limited control of writing conventions (such as 
spelling, grammar/usage, capitalization, punctuation, and/or indentation) makes the writing 
difficult to read. 

Writing is not ratable if completely off topic, completely illegible, written entirely in a language 

other than English, or completely blank. 
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Appendix B 
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Table 2. Michigan HSPT in Communication Arts: Writing Tryout 
Grade 1 1 Enrollment of Participating Schools 
by Region and Size of School 



Region 


Size 


N 


Percent of 
Region 


Percent of 
Total 


1 


Large 


212 


89.08 


3.43 




Small 


26 


10.92 


0.42 


2 


Large 


473 


89.92 


7.66 




Small 


53 


10.08 


0.86 


3 


Large 


348 


27.06 


5.63 




Medium 


771 


59.95 


12.48 




Small 


167 


12.99 


2.70 


4 


Large 


272 


13.07 


4.40 




Medium 


1382 


66.41 


22.37 




Small 


427 


20.52 


6.91 


5 


Large 


1211 


59.19 


19.60 




Medium 


527 


25.76 


8.53 




Small 


308 


15.05 


4.99 


All 




6177 


“ 


100.00 
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Table 3. Michigan HSPT in Communication Arts: Writing Tryout 
Number of Participating Schools 
by Region and Size of School 



Region 


Size 


N 


Percent of 
Region 


Percent of 
Total 


1 


Large 


1 


33.33 


2.44 




Small 


2 


66.67 


4.88 


2 


Large 


3 


60.00 


7.32 




Small 


2 


40.00 


4.88 


3 


Large 


1 


14.29 


2.44 




Medium 


3 


42.86 


7.32 




Small 


3 


42.86 


7.32 


4 


Large 


1 


7.14 


2.44 




Medium 


6 


42.86 


14.63 




Small 


7 


50.00 


17.07 


5 


Large 


4 


33.33 


9.76 




Medium 


4 


33.33 


9.76 




Small 


4 


33.33 


9.76 


All 




41 


— 


100.00 
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Table 4. Michigan HSPT in Communication Arts; Writing Tryout 
Ethnicity of Grade 1 1 Participants by Region and Size of School 





% of 
Total 


3.86 


00 

d 


8.17 


ON 

d 


5.61 


12.71 


1.82 


4.28 


23.93 


cs 

VO 


LV91 


9.08 


5.36 


OOOOl 


White 


% of 
Region 

Group 


98.07 


o 

o 

d 

o 


93.48 


94.12 


89.94 


90.53 


66.67 


87.21 


94.38 


95.93 


78.22 


95.60 


94.63 


89.95 




z 


203 


<s 


430 


48 


295 


699 


96 


225 


1259 


o 


851 


478 


282 


5262 


Hispanic 


% of 
Total 


1.70 




4.55 


0.57 


2.27 


10.23 


5.11 


1.70 


21.59 


4.55 


38.64 


4.55 


4.55 


OOOOl 


% of 
Region 

Group 


1 1.45 




1.74 


961 


1.22 


2.44 


6.25 


911 


2.85 


161 


6.25 


1.60 


2.68 


3.01 


z 


cn 




00 






00 


o\ 


cn 


00 

cn 


00 


00 

VO 


OO 


OO 


VO 


Black 


% of 
Total 






3.64 


0.45 


0.91 


8.64 


15.00 


4.09 


4.09 


160 


1609 


0.45 


160 


OOOOl 


% of 
Region 

Group 






1.74 


961 


190 


2.57 


22.92 


3.49 


0.67 


0.48 


12.32 


0.20 


0.67 


VO 

cn 


z 






00 




<N 


o\ 


cn 

cn 


0\ 


0\ 


<N 


cn 




<N 


220 


Asian 


% of 
Total 


00 

d 




6.45 


00 

d 


19.35 


17.74 


3.23 


16.13 


7.26 


191 


16.94 


6.45 


3.23 


o 

o 

d 

o 


% of 
Region 

Group 


0.48 




1.74 


1.96 


7.32 


2.98 


2.78 


7.75 


L90 


860 


1.93 


091 


1 

1.34 


zvz 


z 






00 




<N 


22 


tT 


20 


0\ 


<N 


<s 


OO 




Tj- 

<N 


American Indian 


% of 
Total 






8.82 




4.41 


16.18 


2.94 


1.47 


21 M 1 


7.35 


20.59 


7.35 


2.94 


OOOOl 


% of 
Region 
Group 






1.30 




160 


1.49 


1.39 


.39 


1.42 


o 

C'l 


1.29 


o 

p 


.67 


91T 


Z 






VO 




cn 




<N 


-- 


o\ 




tT 




<N 


1 68 




. Size 


Large 


Small 


Large 


Small 


Large 


Medium 


Small 


Large 


Medium 


Small 


Large 


Medium 


Small 




Region 






<N 




m 


















All 



LT5 



o:> 

to 
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Tables. Michigan HSPT in Communication Arts: Writing Tryout 
Gender of Grade 1 1 Participants by 
Region and Size of School 





Male 


Female 


Region 


Size 


N 


Percent 

of 

Regional 

Group 


Percent 

of 

Total 


N 


Percent 

of 

Regional 

Group 


Percent 

of 

Total 


1 


Large 


95 


44.81 


3.17 


117 


55.19 


3.73 




Small 


12 


46.15 


0.40 


14 


53.85 


0.45 


2 


Large 


249 


52.98 


8.32 


221 


47.02 


7.04 




Small 


16 


30.19 


0.53 


37 


69.81 


1.18 


3 


Large 


149 


43.19 


4.98 


196 


56.81 


6.24 




Medium 


405 


52.80 


13.53 


362 


47.20 


11.53 




Small 


63 


37.72 


2.10 


104 


62.28 


3.31 


4 


Large 


127 


46.69 


4.24 


145 


53.31 


4.62 




Medium 


677 


49.20 


2232 


699 


50.80 


22.27 




Small 


209 


49.18 


6.98 


261 


50.82 


6.88 


5 


Large 


604 


50.76 


20.18 


586 


49.24 


18.67 




Medium 


251 


48.18 


8.39 


270 


51.82 


8.60 




Small 


136 


44.16 


4.54 


172 


55.84 


5.48 


ALL 




2993 


48.81 


100.00 


3139 


51.19 


100.00 
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Table 7. Michigan HSPT in Communication Arts; Writing Tryout 
Descriptive Statistics for Part 1 



Form 


Mean 


SD 


Interrater 

Reliability 


% Perfect 
Agreement 


% Resolved 


OIC 


2.42 


.88 


.75 


65% 


3% 


02C 


2.31 


.86 


.77 


66 


2 


03C 


2.41 


.91 


.81 


66 


0 


04C 


2.98 


.78 


.75 


69 


1 


05C 


2.36 


.88 


.80 


68 


1 


06C 


3.03 


.72 


.69 


63 


1 


07C 


2.37 


.86 


.73 


65 


2 


08C 


2.54 


.84 


.73 


64 


3 


09C 


2.28 


.89 


.79 


67 


1 


IOC 


2.40 


.85 


.78 


67 


0 



Note: Includes all 10th and 1 1th grade examinees 



Table 8. Michigan HSPT in Communication Arts: Writing Tryout 
Descriptive Statistics for Part 2 



Form 


Mean 


SD 


Interrater 

Reliability 


% Perfect 
Agreement 


% Resolved 


OIC 


2.60 


.85 


.77 


66% 


1% 


02C 


2.26 


.88 


.81 


72 


1 


03C 


2.48 


.83 


.78 


69 


1 


04C 


2.68 


.80 


.72 


66 


2 


05C 


2.30 


.78 


.70 


62 


1 


06C 


2.59 


.83 


.75 


65 


1 


07C 


2.44 


.88 


.76 


63 


1 


08C 


2.66 


.78 


.72 


64 


1 


09C 


2.44 


.87 


.77 


63 ^ 


1 


IOC 


2.61 


.79 


.68 


58 


2 



*Includes all 10th and 1 1th grade examinees 
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Table 9. Michigan HSPT in Communication Arts: Writing Tryout 
Descriptive Statistics for Part 3 



Form 


Mean 


SD 


Interrater 

Reliability 


% Perfect 
Agreement 


% Resolved 


OIC 


2.57 


.82 


.71 


61% 


3% 


02C 


2.42 


.87 


.68 


55 


4 


03C 


2.42 


.83 


.68 


63 


4 


04C 


2.54 


.89 


.73 


60 


“ 3 


05C 


2.31 


.82 


.65 


55 


3 


06C 


2.52 


.86 


.70 


60 


3 


07C 


2.53 


.90 


.73 


60 


4 


08C 


2.72 


.78 


.61 


57 


5 


09C 


2.34 


.91 


.74 


60 


3 


IOC 


2.52 


.77 


.67 


61 


3 




*lncludes all 10th and 1 1th grade examinees 
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Table 10. Michigan HSPT in Conununication Arts: Writing Tryout 
Variance Components and G-Coefficients 



Form 


Persons 


Prompts 


Raters: 

Prompts 


Persons X 
Prompts 


Persons x 
Raters: 
Prompts 


G-Coefficient 


OIC 


.38 


.02 


.00 


.17 


.21 


.80 


02C 


.40 


.01 


.00 


.21 


.21 


.79 


03C 


.34 


.00 


.00 


.25 


.21 


.74 


04C 


.29 


.01 


.00 


.18 


.19 


.76 


05C 


.36 


.00 


.00 


.20 


.21 


.78 


06C 


.27 


.02 


.00 


.16 


.22 


.75 


07C 


.39 


.01 


.01 


.25 


.22 


.77 


08C 


.36 


.02 


.00 


.15 


.22 


, O 
00 


09C 


.43 


.01 


.00 


.21 


.21 


o 

00 


IOC 


.33 


.03 


.00 


.18 


.22 


.78 



Table 11. Michigan HSPT in Communication Arts: Writing Tryout 
Correlations Between Three Test Parts 
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Table 12. Michigan HSPT in Communication Arts: Writing Tryout 
Holistic Score Distribution - Part 1 
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Table 13. Michigan HSPT in Communication Arts; Writing Tryout 
Holistic Score Distribution - Part 2 
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Table 14. Michigan HSPT in Communication Arts: Writing Tryout 
Holistic Score Distribution - Part 3 
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Table 15. Michigan HSPT in Communication Arts: Writing Tryout 
Holistic Score Distribution by Gender - Part 1 
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Table 16. Michigan HSPT in Communication Arts: Writing Tryout 
Holistic Score Distribution by Gender - Part 2 
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Table 17. Michigan HSPT in Communication Arts: Writing Tryout 
Holistic Score Distribution by Gender - Part 3 
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Table 18. Michigan HSPT in Communication Arts: Writing Tryout 
Holistic Score Distribution by Ethnicity* - Part 1 
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Table 19. Michigan HSPT in Communication Arts; Writing Tryout 
Holistic Score Distribution by Ethnicity* - Part 2 
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Table 20. Michigan HSPT in Communication Arts: Writing Tryout 
Holistic Score Distribution by Ethnicity* - Part 3 
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Appendix C 



77 



Table 23. Michigan HSPT in Communication Arts: Writing Pilot 
Grade 11 Enrollment of Participating Schools by 
Region and Size of School 



Region 


Size 


N 


Percent of 
Region 


Percent of 
Total 


1 


Large 


747 


100.00 


5.32 


2 . 


Large 


1056 


100.00 


7.52 


3 


Large 


954 


2 ^ 


6.79 




Medium 


2512 


68.50 


17.89 




Small 


201 


5.48 


1.43 


4 


Medium 


648 


78.64 


4.62 




Small 


176 


21.36 


1.25 


5 


Large 


310 


50.82 


2.21 




Medium 


110 


18.03 


0.78 




Small 


190 


31.15 


1.35 


6 


Medium 


2804 


62.04 


19.97 




Small 


1716 


37.96 


12.22 


7 


Medium 


690 


26.37 


4.91 




Small 


1927 


12 ,. 62 > 


13.72 


All 




14041 


— 


100.00 





78 
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Table 24. Michigan HSPT In Communication Arts: Writing Pilot 
Number of Participating Buildings by 
Region and Size of School 



Region 


Size 


N 


Percent of 
Region 


Percent of 
Total 


1 


Large 


5 


100.00 


4.46 


2 


Large 


6 


100.00 


5.36 


3 


Large 


3 


15.79 


2.68 




Medium 


13 


68.42 


11.61 




Small 


3 


15.79 


2.68 


4 


Medium 


4 


66.67 


3.57 




Small 


2 


33.33 


1.79 


5 


Large 


1 


25.00 


0.89 




Medium 


1 


25.00 


0.89 




Small 


2 


50.00 


1.79 


6 


Medium 


15 


41.67 


13.39 




Small 


2 


50.00 


1.79 


7 


Medium 


6 


16.67 


5.36 




Small 


30 


83.33 


26.79 


All 




112 


-- 


100.00 
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Table 25. Michigan HSPT in Communication Arts; Writing Pilot 
Ethnicity of Grade 1 1 Participants by 
Region and Size of School 





1 White 1 


Total II 
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Table 26. Michigan HSPT in Communication Arts: Writing Pilot 
Gender of Grade 1 1 Participants by 
Region and Size of School 





Male 


Female 


Region 


Size 


N 


Percent 

of 

Regional 

Group 


Percent 

of 

Total 


N 


Percent 

of 

Regional 

Group 


Percent 

of 

Total 


1 


Large 


291 


39.22% 


4.37% 


451 


60.78% 


6.24% 


2 


Large 


468 


44.96 


7.02 


573 


55.04 


7.93 


3 


Large 


451 


47.72 


6.77 


494 


52.28 


6.84 




Medium 


1212 


48.69 


18.19 


1277 


51.31 


17.67 




Small 


91 


45.27 


1.37 


110 


54.73 


1.52 


4 


Medium 


302 


47.48 


4.53 


334 


52.52 


4.62 




Small 


82 


46.59 


1.23 


94 


53.41 


1.30 


5 


Large 


162 


52.43 


2.43 


147 


47.57 


2.03 




Medium 


66 


60.55 


0.99 


43 


39.45 


0.60 




Small 


96 


51.06 


1.44 


92 


48.94 


1.27 


6 


Medium 


1359 


49.19 


20.39 


1404 


50.81 


19.43 




Small 


836 


49.29 


12.55 


860 


50.71 


11.90 


7 


Medium 


324 


47.44 


4.86 


359 


52.56 


4.97 




Small 


924 


48.35 


13.87 


987 


51.65 


13.66 


All 




6664 


47.98 


100.00 


7225 


52.02 


100.00 




O 

ERIC 
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Table 28. Michigan HSPT in Communication Arts: Writing Pilot 
Descriptive Statistics for Part 1 



Form 


Mean 


SD 


Interrater 

Reliability 


% Perfect 


% Resolved 


OIC 


2.33 


.74 


.63 


69% 


2% 


02C 


2.29 


.81 


.71 


69 


2 


03C 


2.49 


.81 


.67 


66 


2 


04C 


2.52 


.78 


.65 


68 


1 


05C 


2.32 


.81 


.71 


70 


1 


06C 


2.34 


.79 


.56 


68 


3 


07C 


2.35 


.85 


.79 


73 


1 


08C 


2.41 


.83 


.70 


66 


2 



Table 29. Michigan HSPT in Communication Arts: Writing Pilot 
Descriptive Statistics for Part 2 



Form 


Mean 


SD 


Interrater 

Reliability 


% Perfect 


% Resolved 


OIC 


2.57 


.77 


.75 


69% 


2% 


02C 


2.43 


.77 


.78 


71 


0 


03C 


2.51 


.78 


.73 


66 


1 


04C 


2.48 


.77 


.77 


71 


0 


05C 


2.42 


.82 


.76 


69 


0 


06C 


2.44 


.72 


.77 


74 


0 


07C 


2.57 


.77 


.73 


70 


1 


08C 


2.46 


.77 


.74 


69 


1 
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Table 30. Michigan HSPT in Communication Arts; Writing Pilot 
Descriptive Statistics for Part 3 




Form 


Mean 


SD 


Interrater 

Reliability 


% Perfect 


% Resolved 


OIC 


2.74 


.76 


.67 


64% 


1% 


02C 


2.63 


.77 


.72 


65 


1 


03C 


2.61 


.77 


.72 


65 


1 


04C 


2.60 


.78 


.73 


68 


1 


05C 


2.53 


.80 


.77 


70 


1 


06C 


2.66 


.75 


.72 


66 


1 


07C 


2.59 


.79 


.75 


67 


0 


08C 


2.57 


.80 


.73 


63 


1 



Table 31. Michigan HSPT in Communication Arts: Writing Pilot 
Variance Components and G-Coefficients 



Form 


Persons 


Prompts 


Raters: 

Prompts 


Persons X 
Prompts 


Persons X 
Raters: 
Prompts 


G-Cofficient 


OIC 


.28 


.01 


.00 


.07 


.03 


.74 


02C 


.33 


.01 


.00 


.06 


.03 


.79 


03C 


.33 


.00 


.00 


.07 


.03 


.78 


04C 


.33 


.00 


.00 


.06 


.03 


.79 


05C 


.39 


.00 


.00 


.06 


.03 


.82 


06C 


.32 


.01 


.00 


.06 


.03 


.79 


07C 


.38 


.01 


.01 


.06 


.03 


.82 


08C 


.37 


.01 


.00 


.06 


.03 


.81 
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Table 32. Michigan HSPT in Communication Arts: Writing Pilot 
Generalizability Analyses Results 
by Form, Gender and Ethnic Group 



Form 


Source 


Females 


Males 


Blacks 


Whites 






Variance 

Estimate 


Standard 

Errors 


Variance 

Estimate 


Standard 

Errors 


Variance 

Estimate 


Standard 

Errors 


Variance 

Estimate 


Standard 

Errors 


OIC 


Persons (P) 


.18 


.01 


.28 


.03 


.18 


.08 


.27 


.02-' 




Tasks (T) 


.01 


.01 


.01 


.01 


.00 


.00 


.01 


.01 




Rater: Tasks (R:T) 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 




Persons X Tasks (PxT) 


.06 


.00 


.08 


.01 


.01 


.01 


.07 


.00 




Persons (R:T) 


.03 


.00 


.03 


.00 


.03 


.01 


.03 


.00 


02C 


Persons (P) 


.26 


.02 


.31 


.03 


.27 


.08 


.34 


.02 




Tasks (T) 


.01 


.01 


.01 


.00 


.04 


.03 


.01 


.01 




Rater: Tasks (R:T) 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 




Persons x Tasks (PxT) 


.06 


.00 


.07 


.01 


.08 


.02 


.06 


.00 




Persons x (R:T) 


.03 


.00 


.03 


.00 


.02 


.00 


.03 


.00 


03C 


Persons (P) 


.24 


.02 


.39 


.04 


.40 


.08 


.32 


.02 




Tasks (T) 


.00 


.00 


.00 


.00 


.01 


.01 


.00 


.00 




Rater: Tasks (R:T) 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 




Persons x Tasks (PxT) 


.06 


.00 


.06 


.01 


.05 


.01 


.06 


.00 




Persons x (R:T) 


.03 


.00 


.03 


.00 


.04 


.00 


.03 


.00 


04C 


Persons (P) 


.27 


.02 


.32 


.02 


.40 


.12 


.32 


.02 




Tasks (T) 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 




Rater: Tasks (R:T) 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 




Persons X Tasks (PxT) 


.05 


.00 


.07 


.00 


.06 


.02 


.06 


.00 




Persons x (R:T) 


.03 


‘ .00 


.03 


.00 


.03 


.00 


.03 


.00 


05C 


Persons (P) 


.32 


.02 


.42 


.04 


.38 


.11 


.38 


.03 




Tasks (T) 


.00 


.00 


.00 


.00 


.02 


.01 


.00 


.00 




Rater: Tasks (R:T) 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 




Persons x Tasks (PxT) 


.06 


.00 


.06 


.01 


.05 


.01 


.06 


.00 




Persons x (R:T) 


.03 


.00 


.03 


.00 


.03 


.00 


.03 


.00 


06C 


Persons (P) 


.26 


.02 


.31 


.03 


.20 


.05 


.33 


.02 




Tasks (T) 


.01 


.01 


.00 


.00 


.01 


.01 


.01 


.01 




Rater: Tasks (R:T) 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 




Persons X Tasks (PxT) 


.05 


.00 


.06 


.01 


.07 


.01 


.05 


.00 




Persons x (R:T) 


.03 


.00 


.03 


.00 


.03 


.00 


.0"3 


.00 


07C 


Persons (P) 


.29 


.02 


.44 


.04 


.41 


.10 


.38 


.03 




Tasks (T) 


.01 


.01 


.00 


.00 


.01 


.01 


.00 


.00 




Rater: Tasks (R:T) 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 




Persons x Tasks (PxT) 


.05 


.00 


.07 


.01 


.06 


.01 


.06 


.00 




Persons x (R:T) 


.03 


.00 


.03 


.00 


.03 


.00 


.03 


.00 


08C 


Persons (P) 


.27 


.03 


.39 


.04 


.33 


.08 


.38 


.03 




Tasks (T) 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 




Rater: Tasks (R:T) 


.00 


.00 


.00 


.00 


.00 


.00 


.00 


.00 




Persons x Tasks (PxT) 


.05 


.00 


.07 


.01 


.05 


.01 


.06 


.00 




Persons x (R:T) 


.03 


.00 


.03 


.00 


.03 


.00 


.03 


.00 
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Table 33. Michigan HSPT in Communication Arts: Writing Pilot 
Generalizability Coefficients 
by Ethnicity and Gender by Form 



Form 


G-Coefficient 

Females 


G-Coefficient 

Males 


G-Coefficient 

Blacks 


G-Coefficient 

Whites 


OIC 


.68 


.72 


.85 


.73 


02C 


.75 


.77 


.73 


.79 


03C 


.73 


.81 


.82 


.77 


04C 


.77 


.78 


.82 


.79 


05C 


.79 


.82 


.82 


.81 


06C 


.77 


.79 


.68 


.80 


07C 


.79 


.82 


.82 


.81 


08C 


.76 


.80 


.81 


.81 



Table 34. Michigan HSPT in Communication Arts: Writing Pilot 
Correlations Between Components 



Form 


Parts 1 and 2 


Parts 2 and 3 


Parts 1 and 3 


OIC 


.50 


.50 


.47 


02C 


.54 


.57 


.53 


03C 


.55 


.55 


.53 


04C 


.57 


.58 


.53 


05C 


.61 


.59 


.58 


06C 


.58 


.61 


.54 


07C 


.59 


.62 


.58 


08C 


.60 


.55 


.60 


Average Correlation 


.57 


.57 


.55 
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Table 35. Michigan HSPT in Communication Arts: Writing Pilot 
Holistic Score Distribution - Part 1 







?orms 


Score 


OIC 


02C 


03C 


04C 


05C 


06C 


07C 


08C 


1 


9.9 


13.5 


8.5 


7.1 


12.3 


12.0 


13.7 


10.8 


1.5 


7.6 


8.7 


6.7 


5.2 


6.9 


7.2 


7.1 


8.1 


2.0 


34.9 


31.8 


29.3 


30.1 


34.7 


32.1 


31.3 


29.6 


2.5 


14.3 


14.5 


14.6 


15.3 


14.7 


14.8 


12.2 


13.1 


3.0 


23.1 


20.5 


23.2 


25.2 


18.0 


22.4 


21.9 


22.0 


3.5 


7.3 


6.0 


11.0 


9.9 


7.3 


7.0 


6.8 


10.2 


4.0 


2.8 


5.1 


6.7 


7.2 


6.0 


4.6 


7.1 


6.2 


N 


776 


888 


802 


1024 


754 


711 


790 


724 


Mean 


2.33 


2.29 


2.49 


2.52 


2.32 


2.34 


2.35 


2.41 


SD 


0.74 


0.81 


0.81 


0.78 


0.81 


0.79 


0.85 


0.83 


Unratable 


923 


828 


837 


899 


891 


843 


866 


747 



*Numbers from Score 1.0 to 4.0 are represented as percents. 



Table 36. Michigan HSPT in Communication Arts: Writing Pilot 
Holistic Score Distribution - Part 2 





Form 


Score 


OIC 


02C 


03C 


04C 


05C 


06C 


07C 


08C 


1 


6.4 


9.6 


7.9 


7.9 


10.6 


7.3 


6.8 


9.5 


1.5 


5.4 


5.6 


6.5 


4.9 


6.9 


4.5 


4.8 


5.1 


2.0 


25.9 


30.4 


26.6 


30.0 


29.4 


35.3 


26.2 


27.6 


2.5 


15.6 


15.8 


16.2 


17.6 


14.9 


13.2 


15.4 L 


17.1 


3.0 


29.5 


26.5 


26.8 


26.5 


22.8 


28.4 


30.5 


27.9 


' 3.5 


10.1 


7.0 


10.1 


6.3 


9.0 


7.6 


8.9 


7.6 


4.0 


7.1 


5.2 


6.0 


6.8 


6.4 


3.7 


7.3 


5.1 


N 


776 


888 


802 


1024 


754 


711 


790 


724 


Mean 


2.57 


2.43 


2.51 


2.48 


2.42 


2.44 


2.57 


2.46 


SD 


0.77 


0.77 


0.78 


0.77 


0.82 


0.72 


0.77 


0.77 


Unratable 


923 


828 


837 


899 


891 


843 


866 


747, 



*Numbers from Score 1.0 to 4.0 are represented as percents. 
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Table 37. Michigan HSPT in Communication Arts: Writing Pilot 
Holistic Score Distribution - Part 3 





Form 


Score 


OIC 


02C 


03C 


04C 


05C 


06C 


07C 


08C 


1 


4.3 


5.7 


5.9 


7.1 


10.1 


5.9 


8.2 


8.8 


1.5 


4.9 


5.9 


6.4 


5.4 


3.8 


4.6 


4.3 


5.7 


2.0 


18.8 


21.7 


22.2 


23.1 


24.4 


20.3 


21.0 


19.6 


2.5 


17.0 


16.7 


16.7 


14.1 


15.1 


16.7 


18.5 


19.9 


3.0 


31.1 


31.0 


30.8 


31.7 


30.1 


33.5 


30.3 


28.2 


3.5 


13.5 


11.7 


11.0 


12.1 


10.6 


12.0 


10.3 


10.1 


4.0 


10.4 


7.3 


7.1 


6.4 


5.8 


7.0 


7.5 


7.7 


N 


776 


888 


802 


1024 


754 


711 


790 


724 


Mean 


2.74 


2.63 


2.61 


2.60 


2.53 


2.66 


2.59 


2.57 


SD 


0.76 


0.77 


0.77 


0.78 


0.80 


0.75 


0.79 


0.80 


Unratable 


923 


828 


837 


899 


891 


843 


866 


747 



*Numbers from Score 1.0 to 4.0 are represented as percents. 
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Table 38. Michigan HSPT in Communication Arts: Writing Pilot 
Holistic Score Distribution by Gender - Part 1 





Form 


U 

00 

o 




3.7 


5-7 II 


od 

<N 


CO 

cd 


28.6 II 


VZl 


8.4 


o 


2.64 


VO 

r- 

d 


335 


S 


VOZ 


11.3 


32.0 


12.3 


12.6 


00 


3.6 


309 


ZVZ 


00 

d 


398 


07C 


Pm 


7.9 


5.4 


32.8 


12.9 


25.3 


VL 


8.6 


466 


6VZ 


o 

00 

d 


364 


s 


22.0 


66 


28.4 


11.2 


16.6 


6.7 


5.1 


313 


2.15 


00 

00 

d 


491 


06C 


Pm 


9.2 


6.3 


28.7 


15.4 


26.5 


08 


6.0 


415 


9VZ 


0.79 


348 




15.9 


8.3 


36.6 


13.8 


16.9 


5.9 


2.8 


290 


2.18 


0.77 


479 


05C 


Pm 


1 6.3 1 


7.7 


33.9 


14.5 


1 21.7 1 


9.0 


6.8 


442 


9VZ 


0.78 


385 




21.0 1 


5.6 


36.1 


15.1 


12.5 


4.9 


4.9 


305 


2.13 


0.82 


495 


04C 


Pm 


4.4 


3.9 


25.4 


15.3 


o 

o 


11.0 


66 


563 


2.68 


0.76 


399 


s 


10.4 1 


6.8 


36.2 


15.0 1 


19.2 


00 


4.0 


453 


2.33 


0.77 


489 


03C 


Pm 


5.6 


5.0 


26.4 


15.9 


1 27.6 1 


VZl 


7.3 


496 


09Z 


0.77 


378 


s 


13.2 


9.8 1 


33.8 


12.5 


15.5 


9.5 


5.7 


296 


2.29 


oo 

d 


448 


02C 


Pm 


00 


8.3 


31.0 


13.5 


25.4 


7.9 


5.8 


519 


2.43 


0.78 


373 


S' 


21.3 


9.3 


32.2 


16.1 


13.7 


3.3 


4.1 


366 


2.09 


o 

oo 

d 


450 


OIC 


Pm 


2.5 


5.2 


33.9 


16.3 


29.6 


9.5 


3.2 


443 


2.53 


990 


402 


s 


20.0 


901 


36.1 


00 


14.5 


4.5 


2.4 


330 


2.07 


0.77 


510 






Score 


- 




q 

cs 


2.5 


3.0 


in 

cd 


4.0 


Z 


II Mean 


P 

CO 


Unratable 



c 

<u 

a 

<u 

cx, 

c/1 

(U 

'4— > 

c 

<L> 

c/i 

a, 



q 

o 

q 



o 

o 

C/D 



4 ) 

I 

3 

* 



o 

a: 



CO 

00 
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Table 39. Michigan HSPT in Communication Arts; Writing Pilot 
Holistic Score Distribution by Gender - Part 2 



u 




r- 

CO 


3.7 II 


I 23.7 II 


00 

vd 


q 

in 

CO 


roi 


vd 


in 

0 


1 2.66 II 


0 

d 


1 335 II 


QO 

O 




17.2 


VL 


32.7 


18.1 


00 


4.2 


2.9 


309 


2.18 


0.77 


398 


07C 




3.0 


1.5 


24.0 


15.7 


38.0 


9.4 


00 


466 


2.73 


890 


364 




12.5 


9.9 


29.7 


15.0 


19.5 


7.7 


00 

id 


313 


2.33 


0.82 


490 


06C 




3.6 


2.4 


29.9 


12.8 


36.9 


10.1 


4.3 


m 


2.62 


890 


348 




12.4 


69 


43.8 


13.8 


16.2 


4.1 


00 


290 


2.19 


0.71 


479 


05C 




1 6.1 


00 


28.7 


14.3 


VLZ 


601 


00 


442 


2.58 


00 

d 


385 




17.4 


9.5 


30.8 


16.1 


16.4 


6.2 


9£ 


305 


2.19 


180 


495 


04C 




1 2.8 


3.7 


24.0 


19.4 


32.1 


8.5 


9.4 


563 


69 Z 


0.71 


399 




14.1 


1 6.4 1 


37.3 


1 15.7 1 


19.2 


00 

CO 


1 3.5 1 


453 


2.22 


1 0.75 


ON 

00 


u 




4.4 


4.2 


24.2 


15.9 


32.3 


11.7 


7.3 


496 


99Z 


0.73 


378 


O 




13.9 


10.5 


1 V6Z 


16.9 


18.2 


VL 


Tj- 


296 


2.26 


0 

00 

d 


448 


02C 




3.7 


3.5 


27.0 


17.0 


32.8 


8.9 


7.3 


519 


2.64 


0.71 


373 




oo 


uo 

00 


35.2 


14.2 


00 


4.4 


2.2 


366 


2.14 


0.76 


450 


OIC 




2.0 


3.2 


VIZ 


15.1 


35.9 


13.8 


8.6 


443 


2.78 


690 1 


402 




12.4 


00 


31.5 


16.1 


21.21 


5.2 


5.2I 


330 


2.31 


0.79 


510 




Score 


- 


1.5 


2.0 


2.5 


3.0 


3.5 


4.0 


Z 


Mean 


SD 


Unratable 



c 

4> 

<u 

Cu 

(/) 

cd 

a 

<u 

c/3 

2 

a. 

2 

2 

cd 

q 

2 

q 

H 

o 

u 

00 

s 

o 

U4 

■4-< 

c/) 

0> 

X) 

S 

o 

Z 

* 



o 

ERIC 
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Table 40. Michigan HSPT in Communication Arts: Writing Pilot 
Holistic Score Distribution by Gender - Part 3 




er|c 
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Table 41. Michigan HSPT in Communication Arts: Writing Pilot 
Holistic Score Distribution by Ethnicity* - Part 1 




<U 
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Ij 

a 



0 
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<u 
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c/3 
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<U 
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cu 
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W) 
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tH 
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<u 
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‘o 

Ph 



c/3 

Vh 

<D 
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Table 42. Michigan HSPT In Communication Arts: Writing Pilot 
Holistic Score Distribution by Ethnicity* - Part 2 






B 

U 

O 



u 




q 

Os 




26.6 


17.1 


in 

cs 


os 

OO 


in 


608 




00 

d 


ON 

n 

in 


QO 

O 


CQ 


q 

o6 


00 


37.5 


16.1 


35.7 


o'oH 


00 


VO 

vn 


2.34 


0.62 


r- 


u 




oo 

vd 




26.2 


14.7 


31.2 


os 


7.0 


099 


2.57 


LLO 


664 


t- 

O 


n 


001 


001 


34.0 1 


16.0 


18.0 


\ 4.0 1 


q 

od 


o 

in 


2.33 


00 

d 


100 


06C 




r- 

r-‘ 


4.0 


34.5 


12.4 


29.4 


CS 

oo 


q 

cd 


571 


2.46 


0.73 


909 


pa 


cn 


OS 

vd 


41.4 


19.5 


21.8 


4.6 


m 

cs 


r- 

00 


2.36 


0.62 


135 


P 




10.4 


r-‘ 


28.5 


14.7 


23.7 


Os 

od 


00 

vd 


999 


2.44 


0.82 


718 


pa 


14.3 


os 

cs 


31.4 


25.7 


14.3 


r- 

in 


q 

in 


in 

m 


2.31 


o 

00 

d 


o 

ON 


u 




r-‘ 


4.7 


28.8 


00 


VLZ 


7.0 


r-' 


903 


2.51 


0.76 


692 


O 


pa 


26.5 


oo 

od 


26.5 


17.6 


OO 

oo 


5.9 


Os 

in 


m 


2.07 


060 


ON 


03C 




q 


r-' 


25.9 


16.7 


26.7 


OO 

OS 


cs 

vd 


645 


2.51 


0.78 


562 


pa 


oo 

00 


5.0 


26.2 


16.2 


23.8 


11.2 


00 

od 


o 

00 


2.55 


£80 


o 

VO 


02C 




q 

ON 


SO 


roe 


16.5 


25.9 


r— 4 


q 

in 


788 


2.43 


0.77 


625 


pa 


q 


5.0 


35.0 


in 


35.0 


in 

cs 


in 


o 


LVZ 


00 

d 


ON 


OIC 




oo 

uS 


OO 

in 


25.0 


15.7 


30.6 


9'0I 


9'9 


■ 709 


2.59 


0.76 


766 


pa 


13.3 


00 


46.7 


o 

d 

<N 


13.3 


vd 


O'O 


in 


2.20 


890 


CS 

00 




Score 


- 




2.0 


in 

cs 


q 

cd 


in 

cd 


4.0 ' 


z 


Mean 


Q 

CO 


Uneatable 



c/3 

<U 

a 

(D 

C/3 

cd 

'O 

<u 

c 

<u 

c/3 

2 

I 

q 

o 
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0 
CO 

a 

* 

c/3 

Ut 

<u 

1 

3 

z 

(L> 

o 

Z 



a 

<u 

-a 

'O 

<u 

2 

4-^ 
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a 

1 
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Table 43. Michigan HSPT in Communication Arts: Writing Pilot 
Holistic Score Distribution by Ethnicity* - Part 3 



O 

ERIC 



Form 


U 




00 




ON 


liMJ 


r 28.9 II 


l6-0ll 


ON 


809 


|l 09‘3 1 


0.79 


ON 

m 

m 


00 

o 


n 


8.9 




28.6 


21.4 


23.2 


5.4 


5.4 1 


VO 


Ot^T 


0.76 




07C 




7.7 


4.4 


603 


17.0 


31.4 


OO 

d 


7.9 


099 


2.62 


0.79 


664 


n 


22.0 


6.0 


q 

OO 


26.0 


20.0 


6.0 


2.0 


o 


2.21 


eso 


100 


06C 




OO 


3.9 


19.4 


OO 

vd 


32.7 


13.3 


00 


in 


2.70 


0.76 


909 




5.7 


10.3 


27.6 


12.6 


35.6 


6.9 


T— 1 


r- 

OO 


2.44 


0.70 


135 


05C 




9.2 
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24.8 


14.6 
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6.0 
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0.79 


OO 

r- 




5.7 


11.4 


20.0 


25.7 


20.0 


11.4 


5.7 
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COMMUNICATION ARTS: WRITING 
Student Survey 



Directions: Listed below are questions about the writing and classroom instruction that may occur in students’ 
school experiences. The Michigan Department of Education is interested in finding out how often your school 
experience has included the following by the end of tenth grade. 

Please read each question carefully, and answer it the BEST that you can. For each question, choose one of the 
answers from the scale below. 



Never A Very Little B Some C A Lot D 



Mark your answer in the row on the answer sheet whose number corresponds with the number of the question you 
are answering. 

Example: 

By the end of the tenth grade, how often did your school experience include: 

1 . talking to a classmate about your writing? 

If “some” were your best answer, you would fill in the circle labeled “C” in row “1” on your answer sheet. 



By the end of tenth grade, how often did your school experience include: 



1 . writing joumals/logs? 

2. writing personal narratives? 

3. informal writing in response to a questions, an idea, or something you’ve read or learned? 

4. informative writing to communicate ideas or information in a letter, lab report, summary...? 

5 . writing essays that raise or answer questions, take a position, interpret, draw conclusions, problem-solve? 

6. writing fiction or plays? 

7. writing poetry? 



8. prewriting (brainstorming, gathering information, planning...)? 

9. drafting (getting ideas down in preliminary form)? 

10. revising (taking another look at your writing, adding, deleting, rearranging, rethinking, rewriting...)? 

1 1 . editing/proofreading (polishing, correcting spelling and mechanical errors...)? 



Kinds of Writing 



Writing Processes 



Working to Improve these Components of Writing 



12. writing content and ideas? 

13. organizing ideas? 

14. writing style (your sentence structure, word choice, and voice as a writer)? 

15. writing conventions (grammatical usage, capitalization, punctuation, spelling, paragraphing...)? 



Writing in Your School 



1 6. writing about a topic or theme? 

17. writing to explain how you write or how you wrote a particular paper? 

18. writing in classes other than English (social studies, science...)? 



Thank You Very Much! 
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Appendix D 



10 ? 



Table 45. Student Survey Response Means in Writing 
(“*” More than 10% of students responded “never.”) 
0 = Nover, 1 = Very Little, 2 = Some, 3 = A Lot 



By the end of tenth grade, how often did your school experience include: 



Number 


Statement 


Mean 


17* 


writing to explain how you write or how you wrote a particular paper 


.99 


6* 


writing fiction or plays? 


1.11 


7* 


writing poetry? 


1.28 


2* 


writing personal narratives? 


1.39 


r 


writing j oumals/logs? 


1.70 


12 


writing content and ideas? 


1.76 


18 


writing in classes other than English (social studies, science...?) 


1.89 


14 


writing style (your sentence structure, word choice, and voice as a 
writer)? 


1.90 


13 


organizing ideas? 


1.90 


5 


writing essays that raise or answer questions, take a position, interpret, 
draw conclusions, problem-solve? 


1.99 


9 


drafting (getting ideas down in preliminary form)? 


2.02 


4 


informative writing to communicate ideas or information in a letter, lab 
report, summary...? 


2.06 


15 


writing conventions (grammatical usage, capitalization, punctuation, 
spelling, paragraphing...)? 


2.10 


3 


informal writing in response to a question, an idea, or something 
you’ve read or learned? 


2.21 


8 


prewriting (brainstorming, gathering information, planning...)? 


2.25 


16 


writing about a topic or theme? 


2.34 


10 


revising (taking another look at your writing, adding, deleting, 
rearranging, rethinking, rewriting...)? 


2.36 


11 


editing/proofreading (polishing, correcting spelling and mechanical 
errors...)? 


2.40 
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Table 47. Teacher Survey - Communication Arts: Writing 






Statements with ^20% Schools Responding NT 

(N=245) 



Statement 



% of Schools 
Responding NT 



7 41% 

18 41% 



7. Types of writing (Genre): Writing plays 

18. Writing in vour schools: Writing to expleun how they write or how they wrote a particular 
paper. 



Statements with ^50% Schools Responding NSI 



In writing, there were no statements to which more than 50% of the schools responded “Not 
Sufficient Instruction” (NSI). The statement that received the highest percentage (32%) of 
schools choosing the NSI response was statement number 24 which referred to Paper z under E 
of the Teacher Survey. 



Part E asked teachers to read four student papers and circle the grade in which students at their 
school would have had sufficient instraction for the level of proficiency demonstrated in the 
paper. Of the four papers, Paper Z was the final and most complex paper. 
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MICHIGAN HIGH SCHOOL PROHCIENCY TEST IN COMMUNICATION ARTS: WRITING 



Tryout and Pilot Technical Report Development Team 

(alphabetically) 



Jane K. Faulds 
Catherine B. Smith 
Jean W. Yan 



Correspondence concerning this report should be addressed to: 

Jean Yan or Catherine Smith 
ME AP Office 

Michigan Department of Education 
P.O. Box 30008 
Lansing, MI 48909 
(517) 373-8393 (O) 

(517) 335-1186 (F) 
yanj @ state.mi.us 
smithcb @ state.mi.us 



This development development team wishes to thank the following people for their time and 
expertise in reviewing this document and providing suggestions and comments: Drs. Sam 
Ewing and Frederica Frost. ACT, as contractor for the development phase of the HSPT in 
Writing, provided data and most statistical analyses used in this report. 

Sandy Koenigsknecht and Yolanda Y. Stephens 
provided excellent support services for this project. 
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